US20080175568A1

Movatterモバイル変換

Info

Publication number: US20080175568A1
Application number: US12/053,894
Authority: US
Inventors: Takuya Kosugi; Shyh-Jye Anthony Chen
Original assignee: Sony Corp; Sony Electronics Inc
Current assignee: Sony Corp; Sony Electronics Inc
Priority date: 2004-01-26
Filing date: 2008-03-24
Publication date: 2008-07-24
Also published as: US7394974B2; US20050163476A1

Abstract

To avoid discontinuities when resuming play of a digital stream that was paused while being viewed live and being simultaneously recorded, the timestamp and digital stream time base discontinuity count associated with the last-viewed frame are used to access the frame in storage that has the same discontinuity count (indicating that it is in the same segment as the last-viewed frame) and a timestamp one greater than the timestamp of the last-viewed frame.

Description

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 10/830,996, filed Apr. 22, 2004, which claims priority from U.S. provisional patent application Ser. No. 60/539,297, filed Jan. 26, 2004.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to television systems.

2. Description of the Related Art

A user of a personal video recorder (PVR) conveniently may record a video stream at the same time the user is viewing the stream live, i.e., as the stream is being received by the user's television system. If the user desires to leave the room but not miss any part of the program being recorded, the user may manipulate a remote control device to cause the system to “pause”, i.e., to freeze a frame on the TV display while continuing to record the program.

When the user wants to resume viewing the program, he again manipulates the remote control device to cause play to resume (e.g., by pressing a “play” button on the remote device). However, current solutions to providing for a smooth transition in the video from “pause” to resumed play have drawbacks, which can be better understood after the following brief discussion.

Digital multimedia streams may be sent to receivers in a format known as Moving Pictures Expert Group (MPEG) standards such as MPEG-1, MPEG-2 (also used for DVD format), MPEG-4 and other block based transform codecs. In MPEG formatting, the data is encoded using MPEG principles, sent to the receiver, and then decoded at the receiver.

As recognized herein, MPEG frames may have timestamps (sometimes referred to as presentation timestamps, or PTS for short) that can be used to identify the frames. As also understood herein, however, a PTS may not be unique, because MPEG streams can undergo time base discontinuities between frame segments, after which PTS numbers can repeat. Accordingly, while a PTS value identifies a frame within a segment, it does not necessarily uniquely identify the frame within the entire stream.

With the above in mind, one way to resume play is simply not to care about the possibility of discontinuity and either resume playing the live feed or resume playing using the recorded feed using, as a best guess of where the video was paused, the PTS of the last live frame that was displayed at the time of pause. For reasons set forth above this can result in undesired discontinuities that hinder viewing enjoyment.

A second way to resume play is to essentially never show the live feed, but rather to always display the video from the recorded stream as though it were the live feed. While this avoids discontinuities because the live feed is never used for display, it results in latency (delays) particularly if the user changes channels. With the above drawbacks in mind, the solutions herein have been provided.

SUMMARY OF THE INVENTION

The invention provides systems and methods for precisely associating and locating viewed digital content such as video frame with stored digital content using a pair of key information, namely, discontinuity counts and timestamps. The invention permits a user to precisely access any preferred video frame. Particular uses of the invention are in pausing and resuming play of a video stream, as well as bookmark operations for later access for viewing or editing. The digital data may be video or audio or other data whose presentation is based on timestamps.

In one aspect the invention precisely associates and locates viewed digital content such as video frames with stored digital content by using a pair of key information, namely, discontinuity count and timestamp (e.g., PTS) information. With the invention disclosed herein, a user may access digital content in precise locations, such as a specified video frame. In non-limiting examples the invention is useful for precise video pause/resume operations, bookmark applications including jumping to a certain video frame which is stored in a storage medium, and video stream editing applications. The invention is useful not only with video data but also audio data and other data, and it may be used to associate any presented digital content with stored data.

Accordingly, a television system includes a processor which associates at least one timestamp of at least one digital information element, such as a video frame, with at least one discontinuity count. A display presents digital data such as video streams from the processor, and a storage stores digital data such as video streams. The timestamp and discontinuity count are used by the processor for resuming play, using data in the storage, of a digital data stream such as a video stream that was paused or bookmarked during live play of the stream. The processor may be associated with a personal video recorder.

The preferred processor uses the timestamp and discontinuity count of a last-presented frame in a live digital stream to resume play using digital data in the storage. Digital frames in the storage are associated with respective timestamps and digital stream discontinuity counts.

In another aspect, a method for providing for smooth resumption of play of a digital stream includes sending the stream to a decoder for live display of the stream on a display. Also, the method includes sending the stream to a storage for storage of the stream, with frames of the stream that are sent to the storage being associated with respective timestamps and digital stream discontinuity counts. A “pause” command can be received, in which case the timestamp and discontinuity count of the paused frame are recorded. Upon receipt of a “resume play” command, using the timestamp and discontinuity count of the paused frame a frame is retrieved from the storage based on the frame having a discontinuity count matching the discontinuity count of the paused frame, and preferably based on the frame having the next sequential timestamp.

In still another aspect, a television system includes a TV display, a decoder driving the display, and means for sending a received digital stream having time base discontinuities through the decoder for presentation thereof on the display. Means are provided for pausing the display. Also, the system includes means for sending a recorded version of the received digital stream through the decoder for presentation thereof on the display upon resumption of play, with the means for sending a recorded version accounting for time base discontinuities in the stream.

BRIEF DESCRIPTION OF THE DRAWINGS

The details of the present invention, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which:

FIG. 1 is a block diagram of the system of the present invention;

FIG. 2 is a flow chart of the present logic;

FIG. 3 is a schematic diagram illustrating the display process using the live video feed prior to entering a “pause” command, showing both logical and hardware system blocks and omitting some components for clarity that are not essential for understandingFIG. 3; and

FIG. 4 is a schematic diagram of the display process using recorded video feed after entering a “pause” command followed by a “resume” command, showing both logical and hardware system blocks and omitting some components for clarity that are not essential for understandingFIG. 4.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring initially toFIG. 1, a television (TV) system is shown, generally designated10, that includes an audio/video TV display12 that conventionally receives, potentially through a processor orprocessors14 that may be housed in a set-top box or personal video recorder (PVR), televised content from an antenna, satellite dish, cable, etc. for display of the content. Theprocessor14 alternatively can be incorporated into the housing of thedisplay12 to function in accordance with the disclosure herein, or it can be implemented by plural processors (e.g., one in a PVR and one in the TV or set-top box) acting in concert with each other. As shown inFIG. 1, the content from theprocessor14 is decoded by adecoder16 in accordance with principles known in the art prior to display. Also, content being viewed live on thedisplay12 may be recorded and stored on astorage18, e.g., an optical or magnetic disk or other memory media that can be part of a PVR.

It is to be understood that the term “television” or “television system” encompasses any apparatus that has a television tuner and the below-described capability in a single housing or in separate housings that cooperate together. For instance, theprocessor14 alternatively can be incorporated into a personal video recorder (PVR) that functions in accordance with the present invention, or even into a standalone computer such as a PC or laptop with its own monitor (not shown), and can communicate with thedisplay12 by wired or wireless link or simply by transferring data from the TV to the computer.

In the preferred non-limiting embodiment shown, theprocessor14 may access one or more software or hardware elements to undertake the present logic.

Additionally, theprocessor14 can be associated with a receiver for conventionally receiving control signals from a portableremote control device20 that functions in accordance with principles known in the art. It is to be understood that whileFIG. 1 shows that thedevice20 can be a conventional TV remote control device, less preferably other devices can be used, such as but not limited to keyboards, keypads, mice, touch screen technology, voice activation/recognition technology, etc. Conventional channel, volume, and TV settings buttons (not shown) can also be provided. Using the remote20, a user can, e.g., play a stream, record a stream, pause a stream, bookmark a location in a stream, etc.

Theprocessor14 executes the logic set forth herein. The flow charts herein illustrate the structure of the logic modules of the present invention as embodied in computer program software. Those skilled in the art will appreciate that the flow charts illustrate the structures of logic elements, such as computer program code elements or electronic logic circuits, that function according to this invention. Manifestly, the invention is practiced in its essential embodiment by a machine component that renders the logic elements in a form that instructs a digital processing apparatus (that is, a computer or microprocessor) to perform a sequence of function steps corresponding to those shown. Internal logic could be as simple as a state machine.

In other words, the present logic may be established as a computer program that is executed by a processor within, e.g., the present microprocessors/servers as a series of computer-executable instructions. In addition to residing on hard disk drives, these instructions may reside, for example, in RAM of the appropriate computer, or the instructions may be stored on magnetic tape, electronic read-only memory, or other appropriate data storage device.

The logic of the present invention can be seen in reference toFIG. 2. Commencing atblock22, a video stream feed is received live from a source of video, e.g., from a cable or satellite transmitter head end or wirelessly from a broadcaster. Atblock24, the live feed of the video stream is sent through thedecoder16 for display on thedisplay12. Also, the video stream is recorded by storing it in thestorage18.

Block

26 indicates that as the stream is stored, both the timestamps (e.g., PTS) of the frames and a count of discontinuities associated with the video segment of each frame are recorded or otherwise associated with the stored frames. More specifically, a counter that may be implemented by theprocessor14 is initialized at zero at the start of receiving the stream to count up by ones for each time base discontinuity it detects in the stream. The counter value at the time a frame is recorded is associated with that frame, along with its timestamp, which may be obtained from the frame header.

In a preferred implementation counters are provided in the live feed (display) line and the recorded video line, so that the discontinuity count of the frame from the live stream being displayed is known, as is the count for all frames being stored in thestorage18. In the event that frames come unaccompanied by timestamps, timestamps may be calculated for the frames using a clock accessible by theprocessor14. This preferred timestamp compensation guarantees that all the recorded frames have respective timestamp entries.

Atblock28, a command to pause the video display may be received when, for example, a user appropriately manipulates theremote control device20. The timestamp of the last-displayed frame along with its time base discontinuity count are recorded.

Atblock30, a command to resume play is received, and the logic then moves to block32 to look up the next frame at which video presentation will be resumed. To do this, the logic matches the discontinuity count of the last-displayed frame from the live video stream feed to the discontinuity count of a segment of frames in thestorage18, and then selects either the frame from the count-matching segment that has the same timestamp as the last-displayed frame or more preferably that has the next sequentially higher timestamp. Less preferably, frames having other timestamps close to the two mentioned above may be selected within the same segment.

Starting with the frame selected atblock32, the video stream stored in thestorage18 is sent to thedecoder16 atblock34 for resuming presentation of the video stream on thedisplay12 using the data that is recorded in thestorage18, without any discontinuities.

The above process is illustrated inFIGS. 3 and 4.Reference numeral36 inFIG. 3 represents the video stream that is received from the video source (referred to herein as the “live feed” or “live data”). Atlogical location38 the live feed is sent both to thestorage18 along a recording path and to thedecoder16 for display along a decoding path. As the video stream is being recorded, blocks40 and42 inFIG. 3 show that a discontinuity counter in the recording path counts time base discontinuities in the stream (by observing such discontinuities in the stream in accordance with MPEG principles known in the art) while a timestamp reporter notes the timestamps in the headers of the frames of the stream. Also, as indicated atlogical state43, the video stream is stored frame by frame, with each frame being associated with its respective timestamp and with the count that thediscontinuity counter42 had when the particular frame was received.

Also, as shown inFIG. 3 a decodingpath discontinuity counter44 andtimestamp reporter46 note the timestamps and discontinuity count as the live feed is displayed. Note that in bothFIG. 3 andFIG. 4 scenarios, the discontinuity counters42,44 at both the decode path and record path are reset to the same value when recording is commenced to guarantee discontinuity counting is synchronized between both paths.

As indicated atlogical state48, the data being displayed may be buffered in thedecoder16 if desired. When a “pause command” (or a bookmark command) is received, as indicated at50 inFIG. 3 the timestamp and discontinuity count of the last-displayed frame (referred to for illustration as a frame having a timestamp of “t0” and discontinuity count of “n”) is recorded.

FIG. 4 shows the same components asFIG. 3 after a command has been received to pause and then to resume play. As indicated at50 inFIG. 4, the last-displayed frame (timestamp=t0, discontinuity count=n) is in thestorage18, as is the next successive video frame (i.e., timestamp=t1, discontinuity count=n). In the preferred embodiment play is resumed by sending, to thedecoder16 fromstorage18, the next successive frame (timestamp=t1) in the same video segment as the last-displayed frame (indicated by having a matching discontinuity count=n).

While the particular SYSTEM AND METHOD FOR ASSOCIATING PRESENTED DIGITAL CONTENT WITHIN RECORDED DIGITAL STREAM AND METHOD FOR ITS PLAYBACK FROM PRECISE LOCATION as herein shown and described in detail is fully capable of attaining the above-described objects of the invention, it is to be understood that it is the presently preferred embodiment of the present invention and is thus representative of the subject matter which is broadly contemplated by the present invention, that the scope of the present invention fully encompasses other embodiments which may become obvious to those skilled in the art, and that the scope of the present invention is accordingly to be limited by nothing other than the appended claims, in which reference to an element in the singular means “at least one”. All structural and functional equivalents to the elements of the above-described preferred embodiment that are known or later come to be known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the present claims. Moreover, it is not necessary for a device or method to address each and every problem sought to be solved by the present invention, for it to be encompassed by the present claims. Furthermore, no element, component, or method step in the present disclosure is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. No claim element herein is to be construed under the provisions of 35 U.S.C. §112, sixth paragraph, unless the element is expressly recited using the phrase “means for”.

Claims

1. A television systems comprising:

at least one processor associating at least one timestamp of at least one digital data element with at least one discontinuity count;

wherein

the timestamp and discontinuity count are used by the processor for resuming play, using data in a storage accessible to the processor, of a digital data stream that was paused and/or bookmarked during live play of the stream.

2. The system ofclaim 1, wherein the processor is associated with a personal video recorder.

3. The system ofclaim 1, wherein the digital data element includes at least one video frame, and the processor uses the timestamp and discontinuity count of a last-presented video frame in a live video stream to resume play using video data in a storage.

4. The system ofclaim 3, wherein video frames in the storage are associated with respective timestamps and video stream discontinuity counts.

5. The system ofclaim 4, wherein if a frame comes unaccompanied by an associated timestamp, a timestamp is calculated for the frame.

6. The system ofclaim 4, comprising a record discontinuity counter in a recording path and a decode discontinuity counter in a decoding path, and the counters are synchronized with each other.

7. A method for providing for smooth resumption of play of a digital stream, comprising:

detecting a “resume play” command;

using a timestamp and discontinuity count of a paused and/or bookmarked frame, retrieving a frame at least partially based on the frame having a discontinuity count matching the discontinuity count of the paused and/or bookmarked frame; and

synchronizing a discontinuity count in a recording path with a discontinuity count in a decoding path.

8. The method ofclaim 7, further comprising retrieving a frame based on the frame having a timestamp that matches the timestamp of the paused frame.

9. The method ofclaim 7, further comprising retrieving a frame based on the frame having a timestamp that is the next sequential timestamp after the timestamp of the paused frame.

10. The method ofclaim 7, comprising using a personal video recorder to execute at least some of the method acts.

11. (canceled)

12. The method ofclaim 7, comprising generating a timestamp for a frame arriving without a timestamp.

13. A television system, comprising:

at least one TV display;

at least one decoder driving the display;

means for sending a received digital stream having time base discontinuities through the decoder for presentation thereof on the display;

means for pausing and/or bookmarking the display;

means for sending a recorded version of the received digital stream through the decoder for presentation thereof on the display upon resumption of play, the means for sending a recorded version accounting for time base discontinuities in the stream.

14. The system ofclaim 13, wherein the means for sending a recorded version operates based at least in part on at least one timestamp of at least one frame of the digital stream.

15. The system ofclaim 14, wherein at least the means for sending a received digital stream and the means for sending a recorded version are embodied by at least one processor.

16. The system ofclaim 15, wherein the processor is associated with a personal video recorder.

17. The system ofclaim 15, wherein the processor uses the timestamp and discontinuity count of a last-presented frame in a live video stream to resume play using video data in a storage local to the processor.

18. The system ofclaim 17, wherein video frames in the storage are associated with respective timestamps and digital stream discontinuity counts.

19. The system ofclaim 14, wherein if a frame comes unaccompanied by an associated timestamp, a timestamp is calculated for the frame.

20. The system ofclaim 13, comprising a record discontinuity counter in a recording path and a decode discontinuity counter in a decoding path, and the counters are synchronized with each other.