This application claims priority from U.S. provisional patent application No.61/806,821 (attorney docket No. P55273Z), filed 3/29/2013, and is incorporated herein by reference.
Detailed Description
Before the present invention is disclosed and described, it is to be understood that this invention is not limited to the particular structures, process steps, or materials disclosed herein, but is extended to equivalents thereof as would be recognized by those ordinarily skilled in the relevant arts. It is also to be understood that the terminology employed herein is used for the purpose of describing particular examples only and is not intended to be limiting. Like reference symbols in the various drawings indicate like elements. The labels provided in the flowcharts and processes are provided for clarity in explaining the steps and operations and do not necessarily indicate a particular order or sequence.
Example embodiments
A preliminary overview of technical embodiments is provided below, and then specific technical embodiments will be described in further detail. This preliminary summary is intended to assist the reader in understanding the technology more quickly, and is not intended to identify key features or essential features of the technology, nor is it intended to limit the scope of the claimed subject matter.
Adaptive multimedia streaming allows different versions of the same multimedia file to be accessed by a mobile device while the multimedia is being streamed. Changes in radio link conditions can reduce or increase the available bandwidth at the mobile device. The ability to "adapt" by changing different versions of a multimedia file while the multimedia file is being run at the mobile device enables the execution of the multimedia file to continue even as bandwidth is reduced.
Current adaptive multimedia streaming standards and specifications, including hypertext transfer protocol (HTTP) -based streaming services, such as progressive download and dynamic adaptive streaming over HTTP (DASH), have limitations that can degrade the quality of the user's experience under certain conditions.
It is generally assumed that all multimedia servers included in a manifest file (manifest file) of a streaming file include all versions and parts of multimedia. This means that servers with partial content of the multimedia stream cannot be used to stream periods to the mobile device that they do not have. In the event that a particular server with all versions and periods of the multimedia stream becomes overloaded and thus unable to deliver content in the appropriate time frame, the mobile device cannot be notified to reduce its download rate from the server to avoid potential retrieval delays or large packet losses of portions of the multimedia.
In addition, where mobile devices share a common limited bandwidth and contend for resources, the presence of multiple multimedia streams to multiple users will cause congestion and reduce the playback experience by having the mobile devices refill their buffers before playback can continue. This is especially true for live activities when a large number of users attempt to obtain the same multimedia streaming content from a server.
Wireless multimedia standard
A number of multimedia standards have been developed that enable multimedia to be transferred to, from, or between mobile computing devices. For example, in streaming video, the third generation partnership project (3GPP) has developed Technical Specification (TS)26.234 (e.g., version 11.0.0) describing a packet-switched streaming service (PSS) based on the real-time streaming protocol (RTSP) for unicast streaming of on-demand or live content. In addition, hypertext transfer protocol (HTTP) -based streaming services, including progressive download and dynamic adaptive streaming over HTTP (DASH), are described in 3GPP TS 26.247 (e.g., release 11.0.0). The 3 GPP-based Multimedia Broadcast Multicast Service (MBMS) specification TS 26.346 (e.g., release 11.0.0) specifies streaming and downloading techniques for multicast/broadcast content distribution. As such, DASH/PSS/MBMS-based mobile computing devices, such as User Equipment (UE), decode and present streamed video at the UE device. Support for the 3GP file format in 3GPP TS 26.244 (e.g., release 11.0.0) is mandatory in all these specifications to support file download and HTTP-based streaming use cases.
One example of a standard for conversational video communication (e.g., video conferencing) is provided in 3GPP TS 26.114 (e.g., 11.0.0). The standard describes the IMS Multimedia Telephony Service (MTSI), which allows advanced multimedia session services and content to be delivered over Internet Protocol (IP) multimedia subsystem (IMS) based networks. IMS is standardized in 3GPP TS 26.140 (e.g., release 11.0.0). A MTSI based transmitter UE terminal may capture and record video and then transmit the video to a MTSI based receiver UE terminal over a 3GPP network. The receiver UE terminal may then decode and render the video. 3GPP TS 26.140 also enables video sharing using multimedia sharing services (MMI), where support for 3GP file formats is provided.
The standards described above are provided as examples of wireless multimedia standards that may be used to transfer multimedia files to, from, and/or between multimedia devices. These examples are not intended to be limiting. Other standards may be used to provide streaming video, conversational video, or video sharing.
Streaming media standard
A more detailed description of HTTP streaming, and in particular the DASH standard, is provided in the context of embodiments of the present invention. The detailed description is not intended to be limiting. As will be further explained in subsequent paragraphs, embodiments of the invention may be used to efficiently transfer multimedia to, from, and/or between mobile devices by enabling the mobile devices, or servers in communication with the mobile devices, to select and/or transfer multimedia having desired energy characteristics. Multimedia may be communicated using standardized or non-standardized communication mechanisms.
Hypertext transfer protocol (HTTP) streaming may be used as a form of multimedia delivery of internet video. In HTTP streaming, a multimedia file may be divided into one or more segments and delivered to a client using the HTTP protocol. HTTP-based delivery can provide reliability and ease of deployment due to the widespread adoption of HTTP and the underlying protocols for HTTP, including Transmission Control Protocol (TCP)/Internet Protocol (IP). HTTP-based delivery may enable simplified streaming services by avoiding Network Address Translation (NAT) and firewall traversal issues. HTTP-based delivery or streaming may also provide the ability to use standard HTTP servers and caches instead of specialized streaming servers. HTTP-based delivery may provide scalability since the state information on the server side is minimized or reduced. Examples of HTTP streaming technologies may include Microsoft IIS smooth streaming, Apple HTTP real-time streaming, and Adobe HTTP dynamic streaming.
DASH is a standardized HTTP streaming protocol. As shown in fig. 1, DASH may specify different formats for Media Presentation Description (MPD) metadata files 102, which may provide information about the structure and different versions of media content representations stored in the server, and the segment formats. The MPD metadata file contains information about the initialization and media segments for the media player (e.g., the media player may look at the initialization segments to determine the container format and media timing information) to ensure mapping of segments to a media presentation timeline for switching and synchronized presentation with other representations. DASH technology has also been standardized by other organizations (e.g., Moving Picture Experts Group (MPEG), open IPTV forum (OIPF), and hybrid broadcast broadband tv (hbbtv)).
A DASH client may receive multimedia content by downloading segments via a series of HTTP request-response processes. DASH may provide the ability to dynamically switch between different bit rate representations of media content as the bandwidth available to the mobile device changes. Thus, DASH may allow for rapid adaptation to changing network and wireless link conditions, user preferences, and device performance, such as display resolution, type of Central Processing Unit (CPU) employed, memory resource availability, etc. Dynamic adaptation of DASH may provide users with better quality of experience (QoE) including shorter startup delay and fewer rebuffering events than other streaming protocols.
In DASH, Media Presentation Description (MPD) metadata 102 may provide information about the structure and different versions of media content representations stored in web/media server 212, as shown in fig. 2 b. In the example shown in fig. 1, MPD metadata is temporally divided into periods having a predetermined length (60 seconds in the present example). Each time period may include multiple adaptation sets 104. Each adaptation set may provide information about one or more media components having a plurality of encoded alternatives. For example, adaptation set 0 in this example may include various differently encoded audio alternatives (such as different bitrates, mono, stereo, surround, etc.). In addition to providing different quality audio for the multimedia presentation over the period ID, the adaptation set may also include audio in different languages. The different alternatives provided in the adaptation set are referred to as representations 106.
In fig. 1, adaptation set 1 is shown to provide video at different bit rates (e.g., 5 megabits per second (Mbps), 2Mbps, 500 kilobits per second (kbps), or trick modes). Trick modes can be used for seeking, fast-forwarding, rewinding, or other changes of position in a multimedia streaming file. Additionally, video may also be available in different formats, such as two-dimensional (2D) or three-dimensional (3D) video. Each representation 106 may include fragment information 108. The clip information may include initialization information 110 and actual media clip data 112. In this example, an MPEG 4(MP4) file may be streamed from a server to a mobile device. Although MP4 is used in this example, a variety of different codecs may be used, as previously described.
The multimedia in the adaptation set may be further divided into smaller segments. In the example of fig. 1, the 60 second video segment of adaptation set 1 is further divided into four sub-segments 112 of 15 seconds each. These examples are not intended to be limiting. The actual length of the adaptation set and each media segment or sub-segment depends on the media type, system requirements, type of potential interference, etc. The actual media segment or sub-segment may have a length of less than 1 second to several minutes.
Fig. 2a provides an example illustration of HTTP Adaptive Streaming (HAS)210 over time. In the initial 30 second period, the client first retrieves segment 208 from the high-quality representation 202. The segment in this example is approximately 10 seconds long. However, this is not intended to be limiting. The segments may be configured to any desired length at the server. In addition, sub-segments may also be downloaded.
The client then retrieves both fragments in the medium quality representation 204. In a second period of 10 seconds duration, the client switches again and retrieves the segments from the low quality representation 206. The client may switch to a lower quality representation due to a change in the quality of the radio link with the multimedia server. In a third period of 20 seconds duration, the client switches back to the medium quality representation 204, as shown in fig. 2 a. The client may continue to request segments from the selected representation throughout the length of the HAS of the multimedia file from the server to the client running on the multimedia device.
As shown in fig. 2b, MPD metadata information may be transmitted to client 220. The client may operate on a mobile device. The mobile device may be a wireless device configured to receive and display streaming media. In one embodiment, the mobile device may perform only a portion of the functions (e.g., receiving streaming media and then transmitting it to another device or display device for presentation). The mobile device may be configured to run a client 220. The client may request the segment using an HTTP GET 240 message or a series of partial GET messages. The client may control the streaming session (e.g., manage on-time requests and smooth play of sequences of segments, or potentially adjust bit rates or other attributes to react to changes in wireless links, device status, or user preferences).
Fig. 2b shows a DASH-based streaming framework. A media encoder 214 in the Web/media server 212 may encode the input media from the audio/video input 210 into a format for storage or streaming. The media segmenter 216 may be used to divide the input media into a series of segments 232, which may be provided to the web server 218. The client 220 may request new data in segments using an HTTP GET message 234 sent to a web server (e.g., an HTTP server).
For example, the web browser 222 of the client 220 may request multimedia content using an HTTP GET message 240. Web server 218 may provide MPD 242 for multimedia content to clients. The MPD may be used to communicate the index of each segment, as well as the corresponding address of the segment, as shown in associated metadata information 252. The Web browser may extract media from the server segment by segment according to MPD 242 shown in 236. For example, the web browser may request the first segment using the HTTP GET URL (frag1req) 244. A Uniform Resource Locator (URL) or universal resource locator may be used to inform the web server which segment 254 the client will request. The Web server may provide the first segment (i.e., segment 1246). For subsequent segments, the web browser can request segment i using HTTP GET URL (frag i req)248, where i is the integer index of the segment. As a result, the web browser may provide segment i 250. The segments may be presented to the client via a media decoder/player 224.
Fig. 3 illustrates a flow of multimedia content 312 between HTTP server 310 and a 3GPP client 338 running on a mobile device (e.g., UE 336). The HTTP server may interact with a public or private network 322 (or the internet) that is in communication with a core network 324 of a Wireless Wide Area Network (WWAN). In one embodiment, the WWAN may be a 3GPP LTE (i.e., rel.11 or 12) based network or an IEEE 802.16 (i.e., 802.16-2009 or 802.16m-2011) based network. The core network may access a wireless network 330, such as an Evolved Packet System (EPS), via a Radio Access Network (RAN) 332. The RAN may provide multimedia content to clients running on the UE via a node (e.g., evolved node b (enb) 334).
QoE-aware adaptive streaming
The quality of experience (QoE) of HTTP Adaptive Streaming (HAS) can be affected by the one or more servers hosting the representations and the corresponding segments. As discussed previously, this specification assumes that all servers (underlying Uniform Resource Locators (URLs)) include all representations and corresponding segments, respectively. This means that servers with only partial content will not be listed in the MPD file. If servers with partial content are listed in the MPD file, the client will not be able to determine that certain servers do not have certain representations or segments before a request is made and not completed by a particular server. When this occurs, the client QoE can drop dramatically due to the delay in retrieving the missing fragments.
The server may have limited operational capabilities. If a particular server becomes overloaded and fails to deliver content in the appropriate time frame, the server cannot notify one or more clients running on the mobile device to reduce their download rate from the server to avoid potential fragment retrieval delays or large packet losses.
In addition, the server may have limited bandwidth. When multiple clients share a common limited bandwidth and contend for resources, the presence of multiple DASH streams to multiple users will cause congestion and reduce the client's playback experience. The reduced ability of the server to provide fragments may result in undesirable re-buffering at the client. This is especially true when a large number of clients attempt to obtain the same DASH content from a server.
According to embodiments of the invention, the server may modify the set of DASH representations in the manifest list (e.g., MPD) that provide the client. The modifications enable the server to communicate information such as available representations and/or segments, available server capacity, and/or available server bandwidth or throughput to the client. The client may then request a representation that the activity is available. If another server with greater capacity or bandwidth is not available, the client may select a representation or segment that will not overload the available server capacity and bandwidth.
The server typically transmits a supported Base URL site that includes a server Internet Protocol (IP) address such as < Base URL > http://192.168.10.10/sintel/,/BaseURL >. In addition to the server IP address, a binary code corresponding to each representation indicating whether the selected representation is available at the server may be included. For example, the representation availability may be communicated using a binary code referred to as an Available Representation Code (ARC). The communication from the server may include an ARC message (e.g., < Base URL ARC ═ 0011001111 "> http://192.168.10.10/sintel/</BaseURL >). This will be described in more detail in subsequent paragraphs.
The ability to communicate the availability of representations may enable the server to dynamically notify the client with updated binary code for available representations. The binary code may be used by the server to limit client requests for representations that would burden the server in terms of capacity and/or throughput. The client may include the updated binary code in its bit rate adaptation logic and only request representations in the active available list. The feedback mechanism allows a client served by the server to make decisions that help avoid congestion problems at the server, thereby increasing QoE at the client device by reducing rebuffering events and increasing the level of presentation that can be delivered to the client.
Representational code
According to one embodiment, a binary code such as ARC may be predetermined for each representation in a manifest file (e.g., MPD file). In one example, each ARC may assign a bit, which may be "0" or "1", referred to as a Representation Access Bit (RAB) for each representation. At runtime, the server may calculate the upload rate of the server for the streaming media provided to the clients and dynamically update the ARC, which is then used to notify each client.
Fig. 4 provides a diagram illustrating an example of an MPD file with an available representation. In this example, the MPD file includes six different representations tagged with representation Identifications (IDs) 0-5. Each representation having a different bit rate. In this example, ID 0 has the lowest bit rate,and ID5 has the highest bit rate (measured in kilobits per second (Kbits/sec)). Each representation ID is also assigned a RAB, where representation ID 0 is assigned to RAB B5ID5 is assigned to RAB B0As shown. Alternative arrangements are also possible, as will be appreciated.
Since an example MPD file contains 6 representations, the corresponding ARC may include 6 bits, or (B)5B4B3B2B1B0) RAB of (1). In this example embodiment, the most significant bit corresponds to the representation with the lowest bit rate, and vice versa. This example is not intended to be limiting. Many different types of code may be used to transfer the ARC from the server to each client.
Fig. 5 provides an example graph of ARC codes used to illustrate selected available bandwidth rates at a server. As can be seen, for each representation at the server, when the corresponding available representation bit is set to a selected binary value (e.g., "0"), access to the representation by the client is disabled. When the available representation bit is set to the opposite binary value (e.g., "1"), it indicates that access is enabled. This allows each client to know which representations are available to the client.
As an example in a subsequent paragraph, code may be used to communicate which representations are available at the server. The code is transmitted in each MPD file. However, the code may be transmitted in other ways at a desired frequency depending on the extent to which server bandwidth or server capacity changes rapidly occur in the HAS system.
In another example, during a streaming session, a server and a client may perform a set of operations to increase the QoE of each client. The server may receive feedback information from each client to calculate the bandwidth to be allocated to each user. The feedback information may contain the average quality perceived by the user during the HAS session, and the number of rebuffering events experienced by the client. In one embodiment, the user perceived quality may be a pre-calculated quality factor associated with each segment that roughly estimates the Mean Opinion Score (MOS) that will be achieved. The pre-computed quality factor may be included in a manifest file, such as an MPD. The algorithm for bandwidth allocation will be discussed further in subsequent paragraphs.
The server may dynamically modify the ARC so that the download rate of one or more clients does not exceed the maximum supported by the bandwidth rate of the server, or the maximum supported by a particular client. The server may then send the updated ARC to the user in response to the user HTTP request. Examples of transferring ARC information include sending ARC information in a manifest file (e.g., MPD), sending ARC information in a custom HTTP header, sending ARC information via a separate radio channel than the radio channel used to transfer HAS, or sending ARC via higher layer signaling. The client may then receive the ARC and then use this information in the client's bit rate adaptation algorithm when making subsequent requests.
The operations described in the preceding paragraphs enable the server to manage the quality of experience for the user. Additional operations may be performed to optimize QoE by the selected approach. The following provides a number of examples for optimizing user experience.
Method for minimizing quality degradation
Dynamically adjusting the ARC at the server may minimize significant degradation in quality at each user. A higher priority may be given to users that would experience a greater decline if the representation were deactivated (i.e., the RAB of the representation was changed from "1" to "0").
A number of additional factors may be considered. For example, the number of rebuffering events for each client may be determined. The difference between the average quality of each client and the average quality of all users combined may be determined. These factors may then be used as credits (credit) that may be given to the client. The credit may reduce the likelihood that other representations for the client are deactivated. Thus, clients with lower QoE than the average QoE may have their QoE boosted relative to other clients.
In one embodiment, the minimum quality degradation (MQR) algorithm may begin with enabling all representations for all clients. In an iterative manner, the MQR algorithm may search for clients that will experience the smallest degradation in quality when the representation is deactivated. In each cycle, if a client has credit, the client is removed as a potential candidate for deactivation as represented in that cycle, and its credit drops by 1. These loops may continue until the server reaches a desired drop in bandwidth or capacity. When the server has excess bandwidth or capacity, the MQR algorithm may activate the representation first for the client with the most credits.
Method of uniform mean mass
In this example embodiment, all clients may be set to almost the same average quality. The server may check whether the total bandwidth requested by the set of clients combined together exceeds the bandwidth available to the server. The representation for the client with the highest average quality may be deactivated if the requested bandwidth exceeds the available bandwidth of the server for the HAS. The average quality may be calculated by each client and sent to the server.
As previously described, the MOS value may be calculated during the pre-processing stage and may be included in a manifest file such as an MPD. Each segment in the MPD is associated with the MOS value. The segment with the higher data rate has a MOS value equal to or greater than the MOS value of the segment with the lower data rate. There is currently an effort to normalize this quality information to parameters in MPDs for the DASH specification. When downloading the HAS, each client may update the average quality, which may be derived by dividing the sum of each MOS value of the download segments by the number of download segments in the HAS.
In one embodiment, the average quality of a client may be replaced by a quality corresponding to the average quality of the maximum representation permitted for that client. The server may then check the total bandwidth requested again and continue to iteratively deactivate the representation for the client with the greatest average quality until the maximum total bandwidth that the set of clients served by the server that may be combined together may request is less than or equal to the bandwidth available to the server of the HAS. Conversely, if the server detects that the total bandwidth requested by the client is less than the total bandwidth available to the server for the HAS, then additional representations may be activated for the client with the least average quality.
Additional algorithms may be developed and used by the server to provide the desired QoE to the clients served by the server by adjusting which representations are available to each client and communicating availability information to each client at the desired frequency.
QoE-aware server
Another example provides functionality 600 of computer circuitry of a server operable to provide hypertext transfer protocol (HTTP) adaptive streaming (as shown in the flow diagram in fig. 6). The functionality may be implemented as a method or the functionality may be executed as instructions on a machine, where the instructions are included on at least one computer readable medium or one non-transitory machine readable storage medium. The computer circuitry can be configured to determine a bandwidth that a server can use to send HTTP adaptive streaming content to a plurality of clients (block 610). The computer circuitry can also be configured to receive HTTP requests from a plurality of clients for representations in a manifest file provided by a server for HTTP adaptive streaming (block 620). The computer circuitry can also be configured to calculate an availability of each representation provided in the manifest file of the server, wherein the availability is calculated based at least in part on the determined bandwidth (at block 630). Additionally, the computer circuitry can be further configured to communicate the availability of each representation from the server to the plurality of clients (block 640).
In one embodiment, the computer circuitry can be configured to communicate availability of each representation as a maximum download rate to each of the plurality of clients to configure each client to request representations having a bit rate less than the maximum download rate. In another embodiment, the computer circuitry can be configured to communicate the availability of each representation as a Representation Access Bit (RAB) for each representation in the manifest file. The computer circuitry can communicate the RAB for each representation as an Available Representation Code (ARC), wherein the ARC is communicated in response to an HTTP request for the representation from the client.
In one embodiment, the computer circuitry can communicate the RABs for each representation as an Available Representation Code (ARC), wherein the ARC is communicated via a separate radio channel than the radio channel used to communicate the HAS, or in a custom HTTP header. In another embodiment, the ARC may be transmitted with the most significant bits corresponding to the representation having the smallest bit rate. Alternatively, the ARC may be transmitted based on a protocol between the server and each client. In another embodiment, the ARC may be embedded in the manifest file of each server to signal each available representation at that server, such that each server stores representations of different bit rates.
The computer circuitry can be further configured to receive quality of experience (QOE) information from each of a plurality of clients, wherein the plurality of clients receive HTTP adaptive streaming from a server; and calculating an availability of each representation for each client based on the determined bandwidth and the QOE of each of the plurality of clients. The QOE information received from each of the multiple clients may be in the form of statistical information sent by the streaming client, such as: the average download rate, average request rate, number of buffering events, hold off time, number of times representing a switch, average quality, as previously described, or another desired metric that may be used to identify the quality of the HAS at the client.
In another embodiment, the computer circuitry can be configured to enable all representations for each of a plurality of clients; iteratively disabling representations for the selected clients based on the QOE received for each client, wherein representations are disabled for ones of the plurality of clients that will experience a minimum QOE drop relative to others of the plurality of clients; and continuing the iterative disabling of the representation until the available bandwidth is sufficient to send the HTTP adaptive streaming content to the plurality of clients at a close quality (close quality).
The computer circuitry can be configured to calculate a credit score for each of the plurality of clients based at least in part on a number of rebuffering events for the received HTTP adaptive stream and a difference between the average quality for each client and an average combined quality for the plurality of clients, and to loop through the iterative disabling of the representation. When the client has credit, the client may be removed as a candidate for deactivation for representation in the loop and the client's credit score is lowered by a selected value. In another embodiment, the computer circuitry can be configured to perform iterative disabling of the representation for a client of the plurality of clients having a highest average quality of experience; and continuing to perform iterative barring of the representation for each client having the highest average quality of experience until the available bandwidth is sufficient to send the HTTP adaptive streaming content to the plurality of clients.
In one example, the computer circuitry can be further configured to calculate, using the selected load balancing algorithm, an availability of each representation for each client of the plurality of clients based on the QOE of each client and the determined bandwidth.
QoE-aware mobile device
Another example provides functionality 700 of computer circuitry of a mobile device operable to provide hypertext transfer protocol (HTTP) adaptive streaming, as illustrated by the flow diagram in fig. 7. The functionality may be implemented as a method or the functionality may be executed as instructions on a machine, where the instructions are included on at least one computer readable medium or one non-transitory machine readable storage medium. The computer circuitry can be configured to receive HTTP adaptive streaming content from a server (block 710). The computer circuitry can also be configured to determine a quality of experience of the received HTTP adaptive streaming content (block 720). The QoE may be an average MOS value as previously described. The average MOS value is typically based on the downloaded content. However, QoE may be extended to combinations of other statistical information such as average download rate, average request rate, number of buffering events, hold off time, number of handovers, average quality (as described above), or other desired metrics. These factors can be combined to produce a single value representing QoE.
The computer circuitry can also be configured to send the QOE to the server (block 730). Additionally, the computer circuitry can be configured to receive availability of a plurality of representations of HTTP adaptive streaming content from the server, wherein the availability of the plurality of representations is based at least in part on the transmitted QOE (block 740), and transmit an HTTP request to the server for at least one representation provided by the server (block 750).
The computer circuitry of the mobile device can also be configured to send an HTTP request for HTTP adaptive streaming content to a server in the manifest file. In one embodiment, the manifest file may be a media presentation description of a dynamic adaptive streaming over HTTP (DASH) adaptation set.
In another embodiment, the computer circuitry can be configured to receive HTTP adaptive streaming content formatted in a dynamic adaptive streaming over HTTP (DASH) format.
The computer circuitry can also be configured to receive, as each representation, an availability of each representation of HTTP adaptive streaming content representing an access bit (RAB). In another embodiment, the RAB for each representation may be received as an Available Representation Code (ARC), where the ARC is received in response to an HTTP request from the mobile device sent for the representation. In one embodiment, an ARC may be received where the least significant bits correspond to the representation with the highest bit rate and the most significant bits correspond to the representation with the lowest bit rate. Alternatively, ARC may be used in the bit rate adaptation algorithm when making subsequent HTTP requests for representations of HTTP adaptive streaming content.
Additionally, the computer circuitry can be configured to determine the QOE based at least in part on a number of rebuffering events occurring at the mobile device while receiving the HTTP adaptive streaming content.
Method for providing variable bit rate adaptive streaming
Another example provides a method 800 for providing variable bit rate adaptive streaming of multimedia from a server to a plurality of clients, as shown in the flow chart in fig. 8. The method may be executed as instructions on a processor of a machine, computer circuitry, or server, where the instructions are included on at least one computer-readable medium, or one non-transitory machine-readable storage medium. The method includes an operation of determining a bandwidth that a server can use to send variable bit rate adaptive streaming content to a plurality of clients (block 810). An additional operation of the method is receiving requests from a plurality of clients for a server-provided representation in a manifest file for variable bit rate adaptive streaming (block 820). An additional operation of the method is to calculate the availability of each representation provided in the manifest file of the server (block 830). The availability may be calculated based at least in part on the determined bandwidth. Additional operations of the method include transmitting the availability of each representation from the server to the plurality of clients (block 840).
The method 800 may further include transmitting, to each client of the plurality of clients, the availability of each representation as a maximum download rate to configure each client to request representations having a bit rate less than the maximum download rate. Alternatively, the availability of each representation may be communicated as a Representation Access Bit (RAB) for each representation in the manifest file. The RAB of each representation may be transmitted as an Available Representation Code (ARC). In one embodiment, the ARC may be transmitted in response to a request for a representation from a client. Alternatively, the ARC code may be transmitted via a separate radio channel instead of the radio channel used to transmit the variable bit rate streaming, or in a custom hypertext transfer protocol (HTTP) header. For example, the ARC may be received in a custom HTTP header in an HTTP response packet for the downloaded segment.
Fig. 9 provides an example illustration of a wireless device, such as a User Equipment (UE), Mobile Station (MS), mobile wireless device, mobile communication device, tablet, handset, or other type of wireless device. A wireless device may include one or more antennas configured to communicate with nodes or transmission stations such as a Base Station (BS), evolved node b (enb), baseband unit (BBU), Remote Radio Head (RRH), Remote Radio Equipment (RRE), Relay Station (RS), Radio Equipment (RE), Remote Radio Unit (RRU), Central Processing Module (CPM), or other type of Wireless Wide Area Network (WWAN) access point. The wireless device may be configured to communicate using at least one wireless communication standard including 3GPP LTE, WiMAX, High Speed Packet Access (HSPA), bluetooth, and WiFi. The wireless devices may communicate using different antennas for each wireless communication standard, or shared antennas for multiple wireless communication standards. The wireless devices may communicate in a Wireless Local Area Network (WLAN), a Wireless Personal Area Network (WPAN), and/or a WWAN. Although an example of a mobile wireless device is provided, the device need not be wireless. Wired devices may also be used for HAS.
Fig. 9 provides an illustration of a microphone and one or more speakers that may be used for audio input and output of a wireless device. The display screen may be a Liquid Crystal Display (LCD) screen, or other types of display screens such as Organic Light Emitting Diode (OLED) displays. The display screen may be configured as a touch screen. The touch screen may use capacitive, resistive, or other types of touch screen technology. The application processor and the graphics processor may be coupled to internal memory to provide processing and display functions. The non-volatile memory port may also be used to provide data input/output options for a user. The non-volatile memory port may also be used to extend the memory functionality of the wireless device. The keyboard may be integrated with the wireless device or wirelessly connected to the wireless device to provide additional user input. A virtual keyboard may also be provided using a touch screen.
The various techniques, and certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangible media, such as floppy diskettes, compact disc read only memories (CD-ROMs), hard drives, non-transitory computer-readable storage media, or any other machine-readable storage medium, wherein, when the program code is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the various techniques. The circuitry may include hardware, firmware, program code, executable code, computer instructions, and/or software. The non-transitory computer readable storage medium may be a computer readable storage medium that does not include a signal. In the case of program code execution on programmable computers, the computing device can include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. The volatile and non-volatile memory and/or storage elements may be Random Access Memory (RAM), erasable programmable read-only memory (EPROM), flash drives, hard drives, solid state drives, or other media for storing electronic data. The nodes and wireless devices may also include a transceiver module (i.e., transceiver), a counter module (i.e., counter), a processing module (i.e., processor), and/or a clock module (i.e., clock) or timer module (i.e., timer). One or more programs that may implement or utilize the various techniques described herein may use an Application Program Interface (API), reusable controls, and the like. These programs may be implemented in a high level procedural or object oriented programming language to communicate with a computer system. However, one or more programs may be implemented in assembly or machine language, as desired. In any case, the language may be a compiled or interpreted language, and combined with hardware implementations.
It should be appreciated that many of the functional units described in this specification have been labeled as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a custom Very Large Scale Integration (VLSI) circuit or gate array, a ready-to-use semiconductor such as a logic chip, transistor, or other discrete component. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
Modules may also be implemented in software for execution by various types of processors. An identified module of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module.
Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations on different storage devices, and may exist, at least partially, as electronic signals on a system or network. These modules may be active or passive, including agents operable to perform desired functions.
Reference throughout this specification to "an example" or "exemplary" means that a particular feature, structure, or characteristic described in connection with the example is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in an example" or the word "exemplary" in various places throughout this specification are not necessarily all referring to the same embodiment.
As used herein, a plurality of items, structural elements, compositional elements, and/or materials may be presented in a common list for convenience. However, these lists should be construed as though each element of the list is individually identified as a distinct and unique element. Thus, no element of the list should be construed as a de facto equivalent of any other element of the same list solely based on their presentation in a common group without indications to the contrary. Additionally, various embodiments and examples of the present invention may be referred to in conjunction with their replacement of various components. It is to be understood that these embodiments, examples, and alternatives are not to be construed as actual equivalents of each other, but are to be construed as separate and autonomous representations of the present invention.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided, such as examples of layouts, distances, network examples, etc., to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, arrangements, and so forth. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
While the foregoing examples are illustrative of the principles of the present invention in one or more particular applications, it will be apparent to those of ordinary skill in the art that numerous modifications in the use and details of the embodiments can be made without departing from the principles and concepts of the invention. Accordingly, the invention is not limited except as defined by the claims set forth below.