US20190075366A1

Movatterモバイル変換

Info

Publication number: US20190075366A1
Application number: US16/180,418
Authority: US
Inventors: Max Haot; Phillip Tomasz Worthington; Mark Kornfilt; Dayananda Nanjundappa; Roger Kapsi
Original assignee: Livestream LLC
Current assignee: Livestream LLC
Priority date: 2007-02-02
Filing date: 2018-11-05
Publication date: 2019-03-07
Also published as: US20200099991A1; US20140181881A1; US8667160B1; US20160309236A1; US20180227643A1

Abstract

One embodiment of the present invention relates to a system for Internet audio/video delivery. Another embodiment of the present invention relates to a method for Internet audio/video delivery.

Description

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 14/192,082, filed Feb. 27, 2014, which is a continuation of U.S. patent application Ser. No. 12/025,360, filed Feb. 4, 2008, which claims the benefit of U.S. Provisional Application Ser. No. 60/887,922, filed Feb. 2, 2007 and U.S. Provisional Application Ser. No. 60/913,402, filed Apr. 23, 2007. Each of the aforementioned applications are incorporated herein by reference in their entirety.

FIELD OF THE INVENTION

One embodiment of the present invention relates to a system for Internet audio/video delivery.

Another embodiment of the present invention relates to a method for Internet audio/video delivery.

For the purposes of describing and claiming the present invention, the term “audio/video content” is intended to refer to audio content and/or video content (including, without limitation, still pictures and/or moving pictures).

Further, for the purposes of describing and claiming the present invention the term “audio/video data element” is intended to refer to audio/video content in the form of a computer-readable data stream or file (including, without limitation, an audio file such as .midi, .wav, .mp3; a still picture file such as .jpg, .tiff, .bmp; a multimedia file such as .mov, .mpg, avi; and/or an animation such as .swf (a file created with ADOBE software). Of note, a given audio/video data element (e.g., a .swf file) may be a composite of other audio/video data elements.

BACKGROUND OF THE INVENTION

Two examples of patent publications related to distributing data over a computer network include the following: U.S. Patent Application Publication 2006/0259607, published Nov. 16, 2006 in the name of O'Neal et al. and U.S. Patent Application Publication 2003/0051051, published Mar. 13, 2003 in the name of O'Neal et al.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a system architecture according to one embodiment of the present invention;

FIG. 2 shows a system architecture according to a conventional live Internet webcast;

FIG. 3 shows a system architecture according to another embodiment of the present invention; and

FIGS. 4A-4F show web browser screenshots according to other embodiments of the present invention.

Among those benefits and improvements that have been disclosed, other objects and advantages of this invention will become apparent from the following description taken in conjunction with the accompanying figures. The figures constitute a part of this specification and include illustrative embodiments of the present invention and illustrate various objects and features thereof.

DETAILED DESCRIPTION OF THE INVENTION

Detailed embodiments of the present invention are disclosed herein; however, it is to be understood that the disclosed embodiments are merely illustrative of the invention that may be embodied in various forms. In addition, each of the examples given in connection with the various embodiments of the invention are intended to be illustrative, and not restrictive. Further, any trademarks, company names and the like referred to in the present application are intended to be illustrative, and not restrictive. Further still, the figures are not necessarily to scale, some features may be exaggerated to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present invention.

Of note, the application may contain some material that may be subject to copyright protection. A disclaimer of copyright with regard to any material owned by another party is hereby made.

In one embodiment the present invention relates to an online service/platform to enable anybody to launch their own 24/7 TV station on the Internet. As opposed to services like Youtube (e.g., where single video clips can be uploaded and viewed), the service/platform of this embodiment enables users to create 24/7 linear channels which can include pre-recorded audio/video content organized in playlists as well as live audio/video content (like a news program, talk show, etc.). The channels may be interactive as viewers may contact the producers to participate in the show (e.g., using the viewer's webcam and/or microphone). Each of the channel producers may have access to a “studio” which allows each producer to control the channel in realtime, such as, for example, by mixing live cameras, creating playlists, controlling graphics like a channel bug, etc. Various embodiments of the present invention thus permit one or more channels to be controlled by groups of people together in realtime.

In another embodiment the present invention relates to an architecture to enable live television/audio/video distribution. In this regard, whereby traditional TV broadcast is done by rendering/mixing a signal at the broadcast station which is then encoded as a finished product and transmitted to viewers as a complete video stream from a single source, various embodiments of the present invention may utilize a distribution system sending to the viewers (e.g., all the viewers) “control messages” (e.g., “play this video now”, “put this graphic there”, “switch to this live camera signal”, etc.). Based upon such control messages a player component (e.g., disposed on a computer associated with each viewer) is then responsible in this embodiment to access the content from one or more sources (e.g., from third party servers), to buffer the content, to assemble the content, to synchronize the content and to play the content seamlessly giving the impression to the viewer of a live stream playing.

Referring now toFIG. 1, a system architecture according to one embodiment of the present invention is shown. As seen in thisFIG. 1,studios101A-101C are provided (while three studios are shown in this example, any desired number of such studios may be provided). In one example, each ofstudios101A-101C may be associated with a given person acting as a producer and each ofstudios101A-101C may comprise software running on a computer. In one specific example, each ofstudios101A-101C may comprise software (such as ADOBE Flash/Flex/ActionScript 3 software) running in a web browser.

Further, each ofstudios101A-101C communicates withchannel server103, which may control one or more channels (of course, any desired number of channels and/or channel servers may be utilized under the present invention). In one example, such communication may be via the Internet. The communication between each ofstudios101A-101C andchannel server103 may comprise: (a) control messages from each ofstudios101A-101C tochannel server103 for use in controlling eachplayer component105A-105C (discussed in more detail below); (b) control messages fromchannel server103 to each ofstudios101A-101C for use in controlling each ofstudios101A-101C (e.g., what to display on a screen of one of the studios based upon what other producers at one or more of the other studios are doing, what to display on a screen of one of the studios based upon what is being displayed at one or more player components); and/or (c) audio/video content to be sent bychannel server103 to one or more ofplayer components105A-105C (discussed in more detail below).

Channel server

103 also communicates (e.g., directly via an intranet and/or indirectly via the Internet) withserver persistence engine107 and channel quality ofservice engine109.

As mentioned above,player components105A-105C are controlled by channel server103 (e.g., through control messages sent fromchannel server103 via the Internet). In one example, each ofplayer components105A-105C may be associated with a given person acting as a viewer and each ofplayer components105A-105C may comprise software running on a computer (while three player components are shown in this example, any desired number of such player components may be provided). In one specific example, each ofplayer components105A-105C may comprise software (such as ADOBE Flash/Flex/ActionScript 3 software) running in a web browser. In another specific example, each player component may comprise a PC, a TV and/or a mobile device (e.g., a mobile phone or PDA). In another specific example, each ofplayer components105A-105C may comprise a quality of service (QOS) engine, a compositing engine and a rendering engine.

Based upon control messages received by each ofplayer components105A-105C from channel server103 (wherein the messages may vary for different player components and/or the messages may be the same for different player components), each ofplayer components105A-105C gets audio/video content from one or more of audio/video content providers111A-111C for display to each respective viewer (while three audio/video content providers are shown in this example, any desired number of such audio/video content providers may be provided). In one example, each ofplayer components105A-105C may communicate with each of audio/video content providers111A-111C via the Internet. In one specific example, the content may comprise video (e.g., one or more live video feeds, one or more recorded video feeds), one or more still images (e.g., jpg, png) and/or audio (e.g., one or more live audio feeds, one or more recorded video feeds, an MP3 stream).

In another example, one or more ofstudios101A-101C may provide its own content (e.g., live audio/video) to one or more ofplayer components105A-105C through channel server103 (e.g., in a similar manner to conventional Internet television without using control messages to direct the player component to get that specific content from a content provider).

In another example, one or more studios and/or one or more player components may comprise software running as a desktop application (e.g., which communicates with the channel server(s) and/or the content provider server(s) over the Internet).

In another example, various embodiments of the present invention may operate in essentially real-time. In one specific example, the studios may control directly and at any moment in time what each client application (each player component) displays. The producer of a channel may, for example, change the text of the ticker, which will in turn send a control message to the channel server, which will propagate it to all client applications (all player components).

In one example, the client device may comprise a client computer.

In another example, the network may comprise the Internet.

In another example, the client software application may access the first and second audio/video data elements via the network.

In another example, the client software application may access the first and second audio/video data elements from a content server (see, e.g., audio/video content providers111A-111C ofFIG. 1).

In another example, the client software application may access the first audio/video data element from a first content server, the client software application may access the second audio/video data element from a second content server, and the first content server may be distinct from the second content server.

In another example, the client software application may access at least one of the first and second audio/video data elements by opening a data stream.

In another example, the client software application may access at least one of the first and second audio/video data elements by downloading.

In another example, a studio software application may be provided, wherein the studio software application may run on a studio computer (see, e.g.,studios101A-101C ofFIG. 1).

In another example, the studio computer and the messaging server may be operatively connected together via the network such that the messaging software application receives at least one message from the studio software application.

In another example, the at least one message received by the client software application from the messaging software application may be based at least in part upon the at least one message received by the messaging software application from the studio software application.

In another example, the messaging, compositing and/or rendering may be carried out in essentially real-time.

In another embodiment of the present invention a system for distributing audio/video convent via a network is provided, comprising: a plurality of client software applications, wherein each of the plurality of client software applications runs on a respective one of a plurality of client devices (see, e.g.,player components105A-105C ofFIG. 1); and at least one messaging software application, wherein the messaging software application runs on a messaging server (see, e.g.,channel server103 ofFIG. 1); wherein each of the plurality of client devices is operatively connected with the messaging server via the network such that each of the plurality of client software applications receives at least one message from the messaging software application; wherein at least one of the messages cause each of the plurality of client software applications to access at least: i) a first audio/video data element and ii) a second audio/video data element; and wherein at least one of the messages cause each of the plurality of client software applications to composite together at least: i) the first audio/video data element and ii) the second audio/video data element and to render at least the composited first and second audio/video data elements at each respective one of the plurality of client devices.

In one example, at least one message received by at least one of the plurality of client software applications may be distinct from at least one message received by at least another of the plurality of client software applications.

In another example, at least one message received by each of the plurality of client software applications may synchronize each of the plurality of client software applications such as to render the composited first and second audio/video data elements in essentially the same state at essentially the same time.

In another example, a plurality of studio software applications may be provided, wherein each of the plurality of studio software applications may run on a respective one of a plurality of studio computers (see, e.g.,studios101A-101C ofFIG. 1).

In another example, each of the plurality of studio computers may be operatively connected with the messaging server via the network such that the messaging software application receives at least one message from each of the plurality of studio software applications.

In another example, at least one message received by each of the plurality of client software applications from the messaging software application may be based at least in part upon at least one message received by the messaging software application from at least one of the plurality of studio software applications.

In another example, a plurality of messaging software applications may be provided, wherein each of the plurality of messaging software application may run on a respective one of a plurality of messaging servers, wherein each of the plurality of client devices may be operatively connected with at least one of the plurality of messaging servers via the network such that each of the plurality of client software applications receives at least one message from at least one of the plurality of messaging software applications, and wherein each of the plurality of studio computers may be operatively connected with at least one of the plurality of messaging servers via the network such that at least one of the plurality of messaging software applications receives at least one message from at least one of the studio software applications.

In another example, the at least one message may be received at a studio software application from the client software application, and the studio software application may run on a studio computer (see, e.g.,studios101A-101C ofFIG. 1).

In another example, at least one message may be received at the studio software application from the client software application via the network.

In another example, the at least one message may be received at the studio software application from the client software application directly.

In another example, the at least one message may be received at the studio software application from the client software application by passing through at least one intermediary server.

In another example, the steps may be carried out in the order recited.

Referring now toFIG. 2, a system architecture according to a conventional live Internet webcast is shown. As seen in this Fig.,production studio200 is utilized to mix audio/video signals fromsources201A-202D atmixer203. The output ofmixer203 is feed tocharacter generator205. The output ofcharacter generator205 is feed toweb encoder207. The output ofweb encoder207 is feed to content delivery network209 (which may include load balancing) and then on toviewers211. Thus, the content is simply assembled at the source and streamed out.

In contrast toFIG. 2,FIG. 3 shows a system architecture according to an embodiment of the present invention in which each ofremote producers301A-301C utilizes associatedstudio303A-303C to communicate via the Internet withbroadcasting platform302 in order to control what each ofviewers305 sees on an associated player component. Each of the player components may receive control messages from broadcastingplatform302, each of the player components may retrieve content from content sources307 (e.g., as directed by the control messages), each of the player components may receive content from one or more ofstudios303A-303C (e.g., via broadcasting platform302) and/or each of the player components may receive content from broadcastingplatform302. Of note,broadcasting platform302 is operatively connected to the Internet to communicate with the studios, the player components and/or the content sources. Similarly, each of the player components (which are not separately shown in the Fig.) is operatively connected to the Internet to communicate withbroadcasting platform302 andcontent sources307.

Referring now toFIG. 4A, a web browser screenshot of a player component according to an embodiment of the present invention is shown. The player component may include one or more of the following:

- Mail—send a message to one or more channel producers
- Call—call one or more channel producers live with the viewer's webcam (a channel producer may put the calling viewers “on the air” live)
- Chat—chat among viewers and producers
- Guide—browse other channels available to the player component
- Full screen

Referring now toFIGS. 4B-4F, web browser screenshots of a studio according to an embodiment of the present invention is shown.

With regard to theseFIGS. 4B-4F, it is noted that use of the studio may, in one example, proceed as follows:

- A producer logs onto a web site with a channel name, a username and a password and clicks “Create New Channel”
- The producer then selects “Configure The Channel” (seeFIG. 4B), which may be a one-time process
- The producer then selects “Get Content” (seeFIG. 4C), which may comprise searching the Internet for images/audio/video from within the studio
- The producer then selects “Create A Storyboard” (seeFIG. 4D), which may comprise dragging the search results from the “Get Content” step into a storyboard within the studio
- The producer then selects “Broadcast Live” (seeFIGS. 4E and 4F), which may comprise using a live mixer or turning on an “auto-pilot”

More particularly, with regard to “Configure The Channel”,FIG. 4B shows a screenshot of how the producer of a newly created channel can pick a broadcast graphics template and customize and preview (not shown) the graphics. Such graphics may include (but not be limited to): Bug; Lower Third; Ticker; Over The Shoulder; Test Card. This configuration of a channel may, in one example, be a one-time process (similar to the way that some blog publishing software may let you choose and configure templates to establish an identity when you first launch your blog).

Further, with regard to “Get Content”,FIG. 4C shows a screenshot of how the producer of a newly created channel can search the Internet for images/audio/video from within the studio. The “Get Content” tab may allow users to easily type keywords and search integrated sources (e.g., AOL Video, Youtube, Google, etc.). The Search results may be streamed in. The user may preview any clip. The user may drag the clips into the user's storyboard in preparation for a live broadcast and/or play list loop. When the user drags the search results into a storyboard, the system of this embodiment may automatically download the files, transcode the files and post the files to the servers of the system (the user may see a progress bar). ThisFIG. 4C also shows some other example sources of audio/video content including (but not limited to): from the web; from a web site; from another channel; from an online store; from the user's computer; from the user's webcam.

Further, with regard to “Create A Storyboard”,FIG. 4D shows a screenshot of how a storyboard may be used as a playlist for the auto-pilot and/or a guide to produce a live broadcast (similar to a script). In one example, the producer can go into the storyboard and: re-order clips; add notes useful to him and his fellow producers; and/or add text for the presenter to say while on camera (similar to a broadcast teleprompter). In addition, collaboration may be provided such that searches and/or storyboards are available in realtime to any producer logged into a given channel (e.g., if one producer adds text or re-orders a storyboard, all the other producers of that channel can seen the changes in realtime).

Further, with regard to “Broadcast Live”,FIG. 4E shows a screenshot of “Live Studio”, where producers mix live (when the channel is in “auto-pilot” mode a playlist may be automatically followed (in one example, the auto-pilot can be seen pressing the various buttons on the studio screen)). Of course, when auto-pilot is not turned on, the producer may mix in realtime.

In any case, in one example the basic process may be as follows:

- The producer loads a storyboard on the left
- The producer drags the clips into the bank for preview
- The producer presses CUE
- When the system is ready and the producer is ready the producer presses “Go Live”
- The audio/video will then mix (cross fade) with the current output
- The producer may then repeat as needed or turn on the auto-pilot as desired

Of note, the bottom-right of thisFIG. 4E shows the following tabs:

- Graphics—type in information (e.g., names and titles) and control how graphics transition on and off the screen
- Sound—sound effects
- Transitions—wipes and effects
- Statistics—details about who is watching
- Record—record your live show for re-broadcast
- Chat—chat between logged-in producers (output shown at the middle top section)

Still referring to “Broadcast Live”,FIG. 4F shows another screenshot of “Live Studio”. As seen in this Fig., the producer has now selected the cameras tab on the left. Here the producer can see the cameras of all of the other producers logged-in for this channel. In one example, a producer can simply drag any webcam on the bank and make it live. In another example, viewers can call the channel. A switchboard tab may allow someone from the production team to select one or more viewers and put the selected viewer(s) in the camera list for live broadcast. A video mail functionality may be provided if a producer does not reply on-time to one or more calls (the viewer may be able to leave a message for future broadcast).

In another example, the architecture may be platform agnostic.

In another example, the architecture may work for cable operators, mobile operators, etc.

In another embodiment the present invention relates to a realtime shared production tool. In this regard, such a realtime shared production tool may comprise capabilities to have producers around the world controlling (e.g., mixing, etc.) together a single video channel in realtime. This may comprise a “virtual, global TV studio”.

In another embodiment the present invention relates to interactivity with viewers. In this regard, viewers may call the channel (e.g., in realtime) using audio/video (e.g., using the viewer's webcam and/or microphone), be selected by a switchboard and then be put through live on the channel (if the producers so decide).

In another example, the present invention may provide a single studio for each channel.

In another example, the present invention may provide a plurality of studios for each channel (in one specific example, there may be a “master” studio that takes precedence over the other studios associated with a given channel; in another specific example, there may be no “master” studio, such that none of the plurality of studios associated with a given channel takes precedence over the other).

Of note, regarding the case of a plurality of studios for each channel, as new studios become associated with a given channel, the new studio(s) may be synchronized with the existing studio(s).

In another example, a player component may comprise software (e.g., which may be run in a web browser) and the player component/browser may run on a desktop computer, a laptop computer, a PDA, a mobile phone, etc.

In another example, a player component may provide to the viewer no facility for a fast forward operation, a rewind operation or the like (similar to a conventional television).

In another example, each channel may be public or private (e.g., accessible only via invitation and/or through the use of a username/password).

In another example, advertisements may be inserted into one or more channels. In one specific example the advertisements may be inserted on a set schedule (e.g., periodically, every 10 minutes). In another specific example, the advertisements may be targeted based upon channel content (e.g., a channel carrying sports content gets sports-related advertising; a channel carrying fashion content gets fashion-related advertising).

In another example, each of the studios and/or the player components may be an online Rich Internet Application built using ADOBE Flash/Flex/ActionScript 3 and no downloads may be required; further, each of the studios and/or the player components may be MAC/PC/LINUX compatible.

In another example, a .swf (a file created with ADOBE Flash/Flex/ActionScript 3 software) may be utilized to allow producers to put broadcast graphics on one or more desired channels.

As described above, one example of the present invention may provide a live streaming protocol (including control messages) for controlling one or more player components from one or more servers (wherein at least some of the content played by the player components(s) comes from source(s) different than the server(s) providing the control messages).

Further, as described above, another example of the present invention may provide for global distributed production/mixing (e.g., multiple studios at distinct remote locations providing control messages for a given channel to control one or more player components).

Further still, as described above, another example of the present invention may provide for quality of service (QoS) measurement and/or control. In one specific example, a channel server may receive feedback from one or more player components regarding when the player component(s) are ready to play content. In another example, a QoS engine may be integrated to ensure that all the player components remain in synchronization (e.g., late-arriving player components are synchronized with player components that are already being used at the time the late-arriving player components start). In another example, the synchronization may utilize state information. In another example, the synchronization may comprise predictive synchronization.

Further still, as described above, another example of the present invention may provide for interactive communication among the studio(s) and the player component(s).

Further still, as described above, another example of the present invention may provide users (e.g., producers) the ability to record instructions for broadcasting (see the “auto-pilot” discussed above) as one or more text files containing the necessary instructions.

Of note, the embodiments described herein may, of course, be implemented using any appropriate computer hardware and/or computer software. In this regard, those of ordinary skill in the art are well versed in the type of computer hardware that may be used (e.g., a mainframe, a mini-computer, a personal computer (“PC”), a network (e.g., an intranet and/or the Internet)), the type of computer programming techniques that may be used (e.g., object oriented programming), and the type of computer programming languages that may be used (e.g., C++, Basic, AJAX, Javascript, ADOBE Flash/Flex/ActionScript 3). The aforementioned examples are, of course, illustrative and not restrictive.

While a number of embodiments of the present invention have been described, it is understood that these embodiments are illustrative only, and not restrictive, and that many modifications may become apparent to those of ordinary skill in the art. For example, certain methods may have been described herein as being “computer implementable” or “computer implemented”. In this regard, it is noted that while such methods can be implemented using a computer, the methods do not necessarily have to be implemented using a computer. Also, to the extent that such methods are implemented using a computer, not every step must necessarily be implemented using a computer. Further, any desired number of studio(s), player component(s), channel server(s) and/or content provider server(s) may be associated with a given channel. Further still, any desired number of channels (and any desired number of associated producer(s) and/or associated viewer(s)) may be provided. Further still, the various steps may be carried out in any desired order (and any desired steps may be added and/or any desired steps may be eliminated).