US20040123231A1 - System and method for annotating multi-modal characteristics in multimedia documents - Google Patents
System and method for annotating multi-modal characteristics in multimedia documentsDownload PDFInfo
- Publication number
- US20040123231A1 US20040123231A1US10/325,061US32506102AUS2004123231A1US 20040123231 A1US20040123231 A1US 20040123231A1US 32506102 AUS32506102 AUS 32506102AUS 2004123231 A1US2004123231 A1US 2004123231A1
- Authority
- US
- United States
- Prior art keywords
- video
- audio
- selection
- multimedia content
- permitting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F15/00—Digital computers in general; Data processing equipment in general
- G06F15/16—Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/169—Annotation, e.g. comment data or footnotes
Definitions
- the present inventionrelates to the computer processing of multimedia files. More specifically, the present invention relates to the manual annotation of multi-modal events, objects, scenes, and audio occurring in multimedia files.
- Multimedia contentis becoming more common both on the World Wide Web and local computers. As the corpus of multimedia content increases, the indexing of features within the content becomes more and more important. Observing both audio and video simultaneously and annotating that observation results in a higher confidence level.
- multimedia annotation systems and methodsthat permit users to observe solely video, video with audio, solely audio, or audio with video and to annotate what has been observed.
- a computer systemwhich has one or more multimedia files that are stored in a working memory.
- the multi-modal annotation processdisplays a user selected multimedia file, permits the selection of a mode or modes to observe the file content, annotates the observations; and saves the annotations in a working memory (such as a MPEG-7 XML file).
- one aspect of the inventionprovides an apparatus for managing multimedia content, the apparatus comprising: an arrangement for supplying multimedia content; an input interface for permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and an arrangement for annotating observations of a selected mode.
- a further aspect of the inventionprovides a method of managing multimedia content, the method comprising the steps of: supplying multimedia content; permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and annotating observations of a selected mode.
- an additional aspect of the inventionprovides a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for managing multimedia content, the method comprising the steps of: supplying multimedia content; permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and annotating observations of a selected mode.
- FIG. 1is a block diagram depicting a multi-modal annotation system.
- FIG. 2is an illustration of a system annotating video scenes, objects, and events.
- FIG. 3is an illustration of a system annotating audio with video.
- FIG. 4is an illustration of a system annotating audio without video.
- FIG. 1is a block diagram of one preferred embodiment of a multi-modal annotation system in accordance with the present invention.
- the multimedia content and previous annotationsare stored on the storage medium 100 .
- a user 130selects a multimedia file via the annotation tool from the storage medium 100 , it is loaded into working memory 110 and portions of it displayed in the annotation tool 120 .
- the user 130may also request that previously saved annotations associated with the current multi-modal file be loaded from the storage medium 100 into working memory 110 .
- the user 100views the multimedia data by making requests through the annotation tool 120 .
- the user 130then annotates his observations and the annotation tool 120 saves these annotations in working memory 110 .
- the usercan at anytime request the annotation tool 120 to save the annotation on the storage medium 100 .
- FIG. 2is an illustration of a system annotating video scenes, objects, and events.
- the multimedia datahas been loaded from the storage medium 100 into working memory 110 .
- a video tab 290has been selected.
- the multimedia videohas been segmented using scene changed detection into shots.
- a shot list window 200displays a portion of the shots in the multimedia.
- the user 130has selected a shot 210 which is highlighted in the shot list window 200 .
- a key frame 220which is a representative shot in the frames of a shot, is preferably displayed.
- the frames of the shotmaybe viewed in the video window 230 using play controls 240 .
- the videocan be viewed with or without audio depending upon the selection of a mute button 250 .
- the user 130may select annotations for this shot by clicking the boxes in events 260 , static scenes 270 , or key objects 280 lists of boxes. Any significant observations which are not contained in the check boxes can be noted in a keywords text box 300 .
- FIG. 3is an illustration of the system annotating audio with video.
- the multimedia datahas been loaded from the storage medium 100 into working memory 110 .
- the audio with video tab 370has been selected.
- the multimedia videohas been segmented using scene change detection into shots.
- the shot list window 200displays a portion of the shots in the multimedia.
- the shot 210 associated with the current audio positionis highlighted in the shot list window 200 .
- the audio datais displayed in the window 390 .
- a segment of audio 340has been delimited for annotation; that is, the limits or bounds of the audio has been fixed for subsequent annotation.
- the video associated with the audiois shown in 230 .
- the audio data display 390is updated to display the current audio data and the video window 230 changes to reflect the current video frame.
- the user 130may observe the Video and simultaneously hear the audio while making audio annotations.
- the user 130preferably uses the buttons 350 to delimit audio segments.
- Check boxes corresponding to the foreground sounds ( 320 ) (the most prominent sounds in the segment) and background sounds ( 330 ) (sounds which are present but are secondary to other sounds)may be checked to indicated sounds heard within the audio segment 340 . Any significant observations which are not contained in the check boxes can be noted in keywords text box 300 .
- FIG. 4is an illustration of the system annotating audio without video.
- the multimedia datahas been loaded from the storage medium 100 into working memory 110 .
- Audio-without-video tab 400has been selected.
- the audio datais displayed in the window 390 .
- a segment of audio 340has been delimited for annotation.
- the audio data display 390is updated to display the current audio data.
- the user 130may only hear the audio while making audio annotations.
- the user 130uses the buttons 350 to delimit audio segments.
- the check boxes for foreground sounds 320 and background sounds 330may be checked to indicate sounds heard within the audio segment 340 . Any significant observations which are not contained in the check boxes can be noted in the keywords text box 300 .
- the present inventionin accordance with at least one presently preferred embodiment, includes an arrangement for supplying multimedia content, an input interface for permitting the selection, for observation, of a mode associated with the multimedia content, and an arrangement for annotating observations of a selected mode.
- these elementsmay be implemented on at least one general-purpose computer running suitable software programs. These may also be implemented on at least one Integrated Circuit or part of at least one Integrated Circuit.
- the inventionmay be implemented in hardware, software, or a combination of both.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- Television Signal Processing For Recording (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- User Interface Of Digital Computer (AREA)
- Management Or Editing Of Information On Record Carriers (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
- The present invention relates to the computer processing of multimedia files. More specifically, the present invention relates to the manual annotation of multi-modal events, objects, scenes, and audio occurring in multimedia files.[0001]
- Multimedia content is becoming more common both on the World Wide Web and local computers. As the corpus of multimedia content increases, the indexing of features within the content becomes more and more important. Observing both audio and video simultaneously and annotating that observation results in a higher confidence level.[0002]
- Existing multimedia tools provide capabilities to annotate either audio or video separately, but not as a whole. (An example of a video-only annotation tool is the IBM MPEG7 Annotation Tool, inventors J. Smith et al., available through [http://]www.alphaworks.ibm.com/tech/videoannex. Other conventional arrangements are described in: Park et al, “iMEDIA-CAT: Intelligent Media Content Annotation Tool”, Proc. International Conference on Inductive Modeling (ICIM) 2001, South Korea, November 2001; and Minka et al., “Interactive Learning using a Society of Models,” Pattern Recognition, Vol. 30, pp. 565, 1997, TR #349.[0003]
- It has long been recognized that annotating video or audio features in isolation results in a less confidence of the identification of the features.[0004]
- In view of the foregoing, a need has been recognized in connection with providing improved systems and methods for observing and annotating multi-modal events, objects, scenes, and audio occurring in multimedia files.[0005]
- In accordance with at least one presently preferred embodiment of the present invention, there are broadly contemplated multimedia annotation systems and methods that permit users to observe solely video, video with audio, solely audio, or audio with video and to annotate what has been observed.[0006]
- In one embodiment, there is provided a computer system which has one or more multimedia files that are stored in a working memory. The multi-modal annotation process displays a user selected multimedia file, permits the selection of a mode or modes to observe the file content, annotates the observations; and saves the annotations in a working memory (such as a MPEG-7 XML file).[0007]
- In summary, one aspect of the invention provides an apparatus for managing multimedia content, the apparatus comprising: an arrangement for supplying multimedia content; an input interface for permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and an arrangement for annotating observations of a selected mode.[0008]
- A further aspect of the invention provides a method of managing multimedia content, the method comprising the steps of: supplying multimedia content; permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and annotating observations of a selected mode.[0009]
- Furthermore, an additional aspect of the invention provides a program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for managing multimedia content, the method comprising the steps of: supplying multimedia content; permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and annotating observations of a selected mode.[0010]
- For a better understanding of the present invention, together with other and further features and advantages thereof, reference is made to the following description, taken in conjunction with the accompanying drawings, and the scope of the invention will be pointed out in the appended claims.[0011]
- FIG. 1 is a block diagram depicting a multi-modal annotation system.[0012]
- FIG. 2 is an illustration of a system annotating video scenes, objects, and events.[0013]
- FIG. 3 is an illustration of a system annotating audio with video.[0014]
- FIG. 4 is an illustration of a system annotating audio without video.[0015]
- FIG. 1 is a block diagram of one preferred embodiment of a multi-modal annotation system in accordance with the present invention. The multimedia content and previous annotations are stored on the[0016]
storage medium 100. When auser 130 selects a multimedia file via the annotation tool from thestorage medium 100, it is loaded intoworking memory 110 and portions of it displayed in theannotation tool 120. At any time, theuser 130 may also request that previously saved annotations associated with the current multi-modal file be loaded from thestorage medium 100 intoworking memory 110. Theuser 100 views the multimedia data by making requests through theannotation tool 120. Theuser 130 then annotates his observations and theannotation tool 120 saves these annotations inworking memory 110. The user can at anytime request theannotation tool 120 to save the annotation on thestorage medium 100. - FIG. 2 is an illustration of a system annotating video scenes, objects, and events. (Simultaneous reference should also be made to FIG. 1.) The multimedia data has been loaded from the[0017]
storage medium 100 intoworking memory 110. A video tab290 has been selected. The multimedia video has been segmented using scene changed detection into shots. Ashot list window 200 displays a portion of the shots in the multimedia. Here, theuser 130 has selected a shot210 which is highlighted in theshot list window 200. A key frame220, which is a representative shot in the frames of a shot, is preferably displayed. In addition, the frames of the shot maybe viewed in the video window230 usingplay controls 240. The video can be viewed with or without audio depending upon the selection of a mute button250. Theuser 130 may select annotations for this shot by clicking the boxes in events260, static scenes270, or key objects280 lists of boxes. Any significant observations which are not contained in the check boxes can be noted in akeywords text box 300. - FIG. 3 is an illustration of the system annotating audio with video. (Simultaneous reference should also be made to FIG. 1.) The multimedia data has been loaded from the[0018]
storage medium 100 intoworking memory 110. The audio with video tab370 has been selected. The multimedia video has been segmented using scene change detection into shots. Theshot list window 200 displays a portion of the shots in the multimedia. The shot210 associated with the current audio position is highlighted in theshot list window 200. The audio data is displayed in thewindow 390. A segment of audio340 has been delimited for annotation; that is, the limits or bounds of the audio has been fixed for subsequent annotation. The video associated with the audio is shown in230. As theuser 130 uses theplay controls 360, theaudio data display 390 is updated to display the current audio data and the video window230 changes to reflect the current video frame. Thus, theuser 130 may observe the Video and simultaneously hear the audio while making audio annotations. Theuser 130 preferably uses thebuttons 350 to delimit audio segments. Check boxes corresponding to the foreground sounds (320) (the most prominent sounds in the segment) and background sounds (330) (sounds which are present but are secondary to other sounds) may be checked to indicated sounds heard within the audio segment340. Any significant observations which are not contained in the check boxes can be noted inkeywords text box 300. - FIG. 4 is an illustration of the system annotating audio without video. (Simultaneous reference should be made to FIG. 1.) The multimedia data has been loaded from the[0019]
storage medium 100 intoworking memory 110. Audio-without-video tab400 has been selected. The audio data is displayed in thewindow 390. A segment of audio340 has been delimited for annotation. As theuser 130 uses the play controls360, the audio data display390 is updated to display the current audio data. Thus, theuser 130 may only hear the audio while making audio annotations. Theuser 130 uses thebuttons 350 to delimit audio segments. The check boxes for foreground sounds320 and background sounds330 may be checked to indicate sounds heard within the audio segment340. Any significant observations which are not contained in the check boxes can be noted in thekeywords text box 300. - It is to be understood that the present invention, in accordance with at least one presently preferred embodiment, includes an arrangement for supplying multimedia content, an input interface for permitting the selection, for observation, of a mode associated with the multimedia content, and an arrangement for annotating observations of a selected mode. Together, these elements may be implemented on at least one general-purpose computer running suitable software programs. These may also be implemented on at least one Integrated Circuit or part of at least one Integrated Circuit. Thus, it is to be understood that the invention may be implemented in hardware, software, or a combination of both.[0020]
- If not otherwise stated herein, it is to be assumed that all patents, patent applications, patent publications and other publications (including web-based publications) mentioned and cited herein are hereby fully incorporated by reference herein as if set forth in their entirety herein.[0021]
- Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be affected therein by one skilled in the art without departing from the scope or spirit of the invention.[0022]
Claims (19)
1. An apparatus for managing multimedia content, said apparatus comprising:
an arrangement for supplying multimedia content;
an input interface for permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and
an arrangement for annotating observations of a selected mode.
2. The apparatus according toclaim 1 , wherein said input interface permits the selection, for observation, of both of the following associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio.
3. The apparatus according toclaim 1 , wherein said input interface additionally permits the selection, for observation, of solely a video portion of multimedia content.
4. The apparatus according toclaim 1 , wherein said input interface additionally permits the selection, for observation, of solely an audio portion of multimedia content.
5. The apparatus according toclaim 1 , wherein said arrangement for supplying multimedia content comprises a working memory which stores multimedia files.
6. The apparatus according toclaim 1 , wherein said input interface is adapted to: first permit the selection of a multimedia file and then permit the selection of said at least one of: an audio portion simultaneously with video; and a video portion simultaneously with audio.
7. The apparatus according toclaim 1 , further comprising a working memory for saving the annotated observations of a selected mode.
8. The apparatus according toclaim 1 , wherein said input interface is adapted to permit the selection, for observation, at least the following mode associated with the multimedia content: a video portion that includes audio.
9. The apparatus according toclaim 8 , wherein said input interface comprises:
an arrangement for permitting the selection, for observation, of a video mode of multimedia content; and
an arrangement for selectably adding audio to the video mode for observation.
10. A method of managing multimedia content, said method comprising the steps of:
supplying multimedia content;
permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and
annotating observations of a selected mode.
11. The method according toclaim 10 , wherein said step of permitting selection comprises permitting the selection, for observation, of both of the following associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio.
12. The method according toclaim 10 , wherein said step of permitting selection additionally comprises permitting the selection the selection, for observation, of solely a video portion of multimedia content.
13. The method according toclaim 10 , wherein step of permitting selection comprises permitting the selection, for observation, of solely an audio portion of multimedia content.
14. The method according toclaim 10 , wherein said step of supplying multimedia content comprises providing a working memory which stores multimedia files.
15. The method according toclaim 10 , wherein said step of permitting selection comprises: first permitting the selection of a multimedia file and then permitting the selection of said at least one of: an audio portion simultaneously with video; and a video portion simultaneously with audio.
16. The method according toclaim 10 , further comprising the step of providing a working memory for saving the annotated observations of a selected mode.
17. The method according toclaim 10 , wherein said step of permitting selection comprises permitting the selection, for observation, at least the following mode associated with the multimedia content: a video portion that includes audio.
18. The method according toclaim 17 , wherein said step of permitting selection comprises:
permitting the selection, for observation, of a video mode of multimedia content; and
thereafter enabling the addition of audio to the video mode for observation.
19. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for managing multimedia content, said method comprising the steps of:
supplying multimedia content;
permitting the selection, for observation, of at least one of the following modes associated with the multimedia content: an audio portion that includes video; and a video portion that includes audio; and
annotating observations of a selected mode.
Priority Applications (10)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US10/325,061US20040123231A1 (en) | 2002-12-20 | 2002-12-20 | System and method for annotating multi-modal characteristics in multimedia documents |
| TW092129818ATW200416600A (en) | 2002-12-20 | 2003-10-27 | System and method for annotating multi-modal characteristics in multimedia documents |
| US10/539,890US7793212B2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
| JP2004563862AJP4347223B2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multimodal characteristics in multimedia documents |
| KR1020057009874AKR100781623B1 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
| AU2003297416AAU2003297416A1 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
| EP03814249AEP1581893A2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
| CA002507248ACA2507248A1 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
| CNA2003801064602ACN1726496A (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
| PCT/US2003/040726WO2004059536A2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US10/325,061US20040123231A1 (en) | 2002-12-20 | 2002-12-20 | System and method for annotating multi-modal characteristics in multimedia documents |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US10/539,890ContinuationUS7793212B2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| US20040123231A1true US20040123231A1 (en) | 2004-06-24 |
Family
ID=32593641
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US10/325,061AbandonedUS20040123231A1 (en) | 2002-12-20 | 2002-12-20 | System and method for annotating multi-modal characteristics in multimedia documents |
| US10/539,890Expired - LifetimeUS7793212B2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US10/539,890Expired - LifetimeUS7793212B2 (en) | 2002-12-20 | 2003-12-19 | System and method for annotating multi-modal characteristics in multimedia documents |
Country Status (9)
| Country | Link |
|---|---|
| US (2) | US20040123231A1 (en) |
| EP (1) | EP1581893A2 (en) |
| JP (1) | JP4347223B2 (en) |
| KR (1) | KR100781623B1 (en) |
| CN (1) | CN1726496A (en) |
| AU (1) | AU2003297416A1 (en) |
| CA (1) | CA2507248A1 (en) |
| TW (1) | TW200416600A (en) |
| WO (1) | WO2004059536A2 (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050027664A1 (en)* | 2003-07-31 | 2005-02-03 | Johnson David E. | Interactive machine learning system for automated annotation of information in text |
| US20060253781A1 (en)* | 2002-12-30 | 2006-11-09 | Board Of Trustees Of The Leland Stanford Junior University | Methods and apparatus for interactive point-of-view authoring of digital video content |
| US20070204238A1 (en)* | 2006-02-27 | 2007-08-30 | Microsoft Corporation | Smart Video Presentation |
| US20070297786A1 (en)* | 2006-06-22 | 2007-12-27 | Eli Pozniansky | Labeling and Sorting Items of Digital Data by Use of Attached Annotations |
| US20080021894A1 (en)* | 2004-12-21 | 2008-01-24 | Styles Thomas L | System and method of searching for story-based media |
| WO2007136870A3 (en)* | 2006-05-19 | 2008-05-29 | Sciencemedia Inc | Document annotation |
| US20090111374A1 (en)* | 2007-10-30 | 2009-04-30 | Lg Electronics Inc. | Mobile terminal processing and transmitting information related to data play and playing data according to information |
| US20100083153A1 (en)* | 2007-12-07 | 2010-04-01 | Jhilmil Jain | Managing Multimodal Annotations Of An Image |
| US20120159329A1 (en)* | 2010-12-16 | 2012-06-21 | Yahoo! Inc. | System for creating anchors for media content |
| US12386891B2 (en)* | 2021-09-18 | 2025-08-12 | Beijing Baidu Netcom Science Technology Co., Ltd. | Information search method and device, electronic device, and storage medium |
Families Citing this family (35)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060184553A1 (en)* | 2005-02-15 | 2006-08-17 | Matsushita Electric Industrial Co., Ltd. | Distributed MPEG-7 based surveillance servers for digital surveillance applications |
| DE102005022550A1 (en)* | 2005-05-17 | 2006-11-23 | Siemens Ag | Method for postprocessing at least one film shot created during an investigation |
| DE102005025903A1 (en)* | 2005-06-06 | 2006-12-28 | Fm Medivid Ag | Device for annotating motion pictures in the medical field |
| US8805929B2 (en)* | 2005-06-20 | 2014-08-12 | Ricoh Company, Ltd. | Event-driven annotation techniques |
| US8015482B2 (en)* | 2005-09-14 | 2011-09-06 | Microsoft Corporation | Dynamic anchoring of annotations to editable content |
| US7739304B2 (en)* | 2007-02-08 | 2010-06-15 | Yahoo! Inc. | Context-based community-driven suggestions for media annotation |
| US7788099B2 (en)* | 2007-04-09 | 2010-08-31 | International Business Machines Corporation | Method and apparatus for query expansion based on multimodal cross-vocabulary mapping |
| US8583267B2 (en)* | 2007-08-17 | 2013-11-12 | The Invention Science Fund I, Llc | Selective invocation of playback content supplementation |
| US7733223B2 (en)* | 2007-08-17 | 2010-06-08 | The Invention Science Fund I, Llc | Effectively documenting irregularities in a responsive user's environment |
| US8990400B2 (en)* | 2007-08-17 | 2015-03-24 | The Invention Science Fund I, Llc | Facilitating communications among message recipients |
| US8140973B2 (en)* | 2008-01-23 | 2012-03-20 | Microsoft Corporation | Annotating and sharing content |
| US20090319883A1 (en)* | 2008-06-19 | 2009-12-24 | Microsoft Corporation | Automatic Video Annotation through Search and Mining |
| US8839327B2 (en) | 2008-06-25 | 2014-09-16 | At&T Intellectual Property Ii, Lp | Method and apparatus for presenting media programs |
| US9015778B2 (en) | 2008-06-25 | 2015-04-21 | AT&T Intellectual Property I. LP | Apparatus and method for media on demand commentaries |
| US9276761B2 (en)* | 2009-03-04 | 2016-03-01 | At&T Intellectual Property I, L.P. | Method and apparatus for group media consumption |
| US8275623B2 (en) | 2009-03-06 | 2012-09-25 | At&T Intellectual Property I, L.P. | Method and apparatus for analyzing discussion regarding media programs |
| US8380866B2 (en) | 2009-03-20 | 2013-02-19 | Ricoh Company, Ltd. | Techniques for facilitating annotations |
| US8266652B2 (en) | 2009-10-15 | 2012-09-11 | At&T Intellectual Property I, L.P. | Apparatus and method for transmitting media content |
| US9830605B2 (en) | 2009-10-30 | 2017-11-28 | At&T Intellectual Property I, L.P. | Apparatus and method for product marketing |
| KR20110047768A (en) | 2009-10-30 | 2011-05-09 | 삼성전자주식회사 | Apparatus and method for playing multimedia content |
| DE102009060687A1 (en)* | 2009-11-04 | 2011-05-05 | Siemens Aktiengesellschaft | Method and device for computer-aided annotation of multimedia data |
| US8224756B2 (en) | 2009-11-05 | 2012-07-17 | At&T Intellectual Property I, L.P. | Apparatus and method for managing a social network |
| US8760469B2 (en) | 2009-11-06 | 2014-06-24 | At&T Intellectual Property I, L.P. | Apparatus and method for managing marketing |
| US8316303B2 (en)* | 2009-11-10 | 2012-11-20 | At&T Intellectual Property I, L.P. | Method and apparatus for presenting media programs |
| US9031379B2 (en)* | 2009-11-10 | 2015-05-12 | At&T Intellectual Property I, L.P. | Apparatus and method for transmitting media content |
| US8387088B2 (en) | 2009-11-13 | 2013-02-26 | At&T Intellectual Property I, Lp | Method and apparatus for presenting media programs |
| US10708663B2 (en) | 2009-11-13 | 2020-07-07 | At&T Intellectual Property I, L.P. | Apparatus and method for media on demand commentaries |
| US8839306B2 (en) | 2009-11-20 | 2014-09-16 | At&T Intellectual Property I, Lp | Method and apparatus for presenting media programs |
| US9100550B2 (en)* | 2009-11-20 | 2015-08-04 | At&T Intellectual Property I, L.P. | Apparatus and method for managing a social network |
| US8373741B2 (en) | 2009-11-20 | 2013-02-12 | At&T Intellectual Property I, Lp | Apparatus and method for collaborative network in an enterprise setting |
| US9094726B2 (en) | 2009-12-04 | 2015-07-28 | At&T Intellectual Property I, Lp | Apparatus and method for tagging media content and managing marketing |
| US20140164366A1 (en)* | 2012-12-12 | 2014-06-12 | Microsoft Corporation | Flat book to rich book conversion in e-readers |
| US10318572B2 (en)* | 2014-02-10 | 2019-06-11 | Microsoft Technology Licensing, Llc | Structured labeling to facilitate concept evolution in machine learning |
| US10438499B2 (en)* | 2014-08-01 | 2019-10-08 | International Business Machines Corporation | Identification of comprehension burden in multimedia content |
| US12249147B2 (en)* | 2021-03-11 | 2025-03-11 | International Business Machines Corporation | Adaptive selection of data modalities for efficient video recognition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5524193A (en)* | 1991-10-15 | 1996-06-04 | And Communications | Interactive multimedia annotation method and apparatus |
| US5583980A (en)* | 1993-12-22 | 1996-12-10 | Knowledge Media Inc. | Time-synchronized annotation method |
| US5600775A (en)* | 1994-08-26 | 1997-02-04 | Emotion, Inc. | Method and apparatus for annotating full motion video and other indexed data structures |
| US6332144B1 (en)* | 1998-03-11 | 2001-12-18 | Altavista Company | Technique for annotating media |
| US6546405B2 (en)* | 1997-10-23 | 2003-04-08 | Microsoft Corporation | Annotating temporally-dimensioned multimedia content |
Family Cites Families (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5148154A (en)* | 1990-12-04 | 1992-09-15 | Sony Corporation Of America | Multi-dimensional user interface |
| US5442744A (en)* | 1992-04-03 | 1995-08-15 | Sun Microsystems, Inc. | Methods and apparatus for displaying and editing multimedia information |
| US5838313A (en)* | 1995-11-20 | 1998-11-17 | Siemens Corporate Research, Inc. | Multimedia-based reporting system with recording and playback of dynamic annotation |
| US6154600A (en)* | 1996-08-06 | 2000-11-28 | Applied Magic, Inc. | Media editor for non-linear editing system |
| JPH10177511A (en) | 1996-12-18 | 1998-06-30 | Fuji Xerox Co Ltd | File system |
| US6006241A (en)* | 1997-03-14 | 1999-12-21 | Microsoft Corporation | Production of a video stream with synchronized annotations over a computer network |
| US7111009B1 (en)* | 1997-03-14 | 2006-09-19 | Microsoft Corporation | Interactive playlist generation using annotations |
| US6449653B2 (en)* | 1997-03-25 | 2002-09-10 | Microsoft Corporation | Interleaved multiple multimedia stream for synchronized transmission over a computer network |
| US6484156B1 (en)* | 1998-09-15 | 2002-11-19 | Microsoft Corporation | Accessing annotations across multiple target media streams |
| US6956593B1 (en)* | 1998-09-15 | 2005-10-18 | Microsoft Corporation | User interface for creating, viewing and temporally positioning annotations for media content |
| US7647555B1 (en) | 2000-04-13 | 2010-01-12 | Fuji Xerox Co., Ltd. | System and method for video access from notes or summaries |
| US8028314B1 (en)* | 2000-05-26 | 2011-09-27 | Sharp Laboratories Of America, Inc. | Audiovisual information management system |
- 2002
- 2002-12-20USUS10/325,061patent/US20040123231A1/ennot_activeAbandoned
- 2003
- 2003-10-27TWTW092129818Apatent/TW200416600A/enunknown
- 2003-12-19AUAU2003297416Apatent/AU2003297416A1/ennot_activeAbandoned
- 2003-12-19KRKR1020057009874Apatent/KR100781623B1/ennot_activeExpired - Fee Related
- 2003-12-19CNCNA2003801064602Apatent/CN1726496A/enactivePending
- 2003-12-19USUS10/539,890patent/US7793212B2/ennot_activeExpired - Lifetime
- 2003-12-19JPJP2004563862Apatent/JP4347223B2/ennot_activeExpired - Lifetime
- 2003-12-19CACA002507248Apatent/CA2507248A1/ennot_activeAbandoned
- 2003-12-19EPEP03814249Apatent/EP1581893A2/ennot_activeWithdrawn
- 2003-12-19WOPCT/US2003/040726patent/WO2004059536A2/enactiveApplication Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5524193A (en)* | 1991-10-15 | 1996-06-04 | And Communications | Interactive multimedia annotation method and apparatus |
| US5583980A (en)* | 1993-12-22 | 1996-12-10 | Knowledge Media Inc. | Time-synchronized annotation method |
| US5600775A (en)* | 1994-08-26 | 1997-02-04 | Emotion, Inc. | Method and apparatus for annotating full motion video and other indexed data structures |
| US6546405B2 (en)* | 1997-10-23 | 2003-04-08 | Microsoft Corporation | Annotating temporally-dimensioned multimedia content |
| US6332144B1 (en)* | 1998-03-11 | 2001-12-18 | Altavista Company | Technique for annotating media |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060253781A1 (en)* | 2002-12-30 | 2006-11-09 | Board Of Trustees Of The Leland Stanford Junior University | Methods and apparatus for interactive point-of-view authoring of digital video content |
| US8645832B2 (en)* | 2002-12-30 | 2014-02-04 | The Board Of Trustees Of The Leland Stanford Junior University | Methods and apparatus for interactive map-based analysis of digital video content |
| US20050027664A1 (en)* | 2003-07-31 | 2005-02-03 | Johnson David E. | Interactive machine learning system for automated annotation of information in text |
| US7797328B2 (en)* | 2004-12-21 | 2010-09-14 | Thomas Lane Styles | System and method of searching for story-based media |
| US20080021894A1 (en)* | 2004-12-21 | 2008-01-24 | Styles Thomas L | System and method of searching for story-based media |
| US20070204238A1 (en)* | 2006-02-27 | 2007-08-30 | Microsoft Corporation | Smart Video Presentation |
| WO2007136870A3 (en)* | 2006-05-19 | 2008-05-29 | Sciencemedia Inc | Document annotation |
| US20070297786A1 (en)* | 2006-06-22 | 2007-12-27 | Eli Pozniansky | Labeling and Sorting Items of Digital Data by Use of Attached Annotations |
| US8301995B2 (en)* | 2006-06-22 | 2012-10-30 | Csr Technology Inc. | Labeling and sorting items of digital data by use of attached annotations |
| US20090111374A1 (en)* | 2007-10-30 | 2009-04-30 | Lg Electronics Inc. | Mobile terminal processing and transmitting information related to data play and playing data according to information |
| EP2056214A3 (en)* | 2007-10-30 | 2013-03-20 | LG Electronics Inc. | Mobile terminals for sending and receiving additional information linked to time stamps in presented data. |
| US8611963B2 (en) | 2007-10-30 | 2013-12-17 | Lg Electronics Inc. | Mobile terminal processing and transmitting information related to data play and playing data according to information |
| US20100083153A1 (en)* | 2007-12-07 | 2010-04-01 | Jhilmil Jain | Managing Multimodal Annotations Of An Image |
| US8898558B2 (en) | 2007-12-07 | 2014-11-25 | Hewlett-Packard Development Company, L.P. | Managing multimodal annotations of an image |
| US20120159329A1 (en)* | 2010-12-16 | 2012-06-21 | Yahoo! Inc. | System for creating anchors for media content |
| US12386891B2 (en)* | 2021-09-18 | 2025-08-12 | Beijing Baidu Netcom Science Technology Co., Ltd. | Information search method and device, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| KR100781623B1 (en) | 2007-12-07 |
| EP1581893A2 (en) | 2005-10-05 |
| WO2004059536A3 (en) | 2004-12-29 |
| US7793212B2 (en) | 2010-09-07 |
| WO2004059536A2 (en) | 2004-07-15 |
| CN1726496A (en) | 2006-01-25 |
| KR20050085251A (en) | 2005-08-29 |
| US20060218481A1 (en) | 2006-09-28 |
| JP4347223B2 (en) | 2009-10-21 |
| CA2507248A1 (en) | 2004-07-15 |
| AU2003297416A1 (en) | 2004-07-22 |
| TW200416600A (en) | 2004-09-01 |
| JP2006512007A (en) | 2006-04-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7793212B2 (en) | System and method for annotating multi-modal characteristics in multimedia documents | |
| US10031649B2 (en) | Automated content detection, analysis, visual synthesis and repurposing | |
| US7506262B2 (en) | User interface for creating viewing and temporally positioning annotations for media content | |
| US7725829B1 (en) | Media authoring and presentation | |
| US7181757B1 (en) | Video summary description scheme and method and system of video summary description data generation for efficient overview and browsing | |
| US6546405B2 (en) | Annotating temporally-dimensioned multimedia content | |
| US6636238B1 (en) | System and method for linking an audio stream with accompanying text material | |
| US8739040B2 (en) | Multimedia visualization and integration environment | |
| US8930308B1 (en) | Methods and systems of associating metadata with media | |
| US20070079321A1 (en) | Picture tagging | |
| US20070011206A1 (en) | Interactive playlist generation using annotations | |
| JP2006155384A (en) | Video comment input / display method, apparatus, program, and storage medium storing program | |
| KR20030007727A (en) | Automatic video retriever genie | |
| JP3574606B2 (en) | Hierarchical video management method, hierarchical management device, and recording medium recording hierarchical management program | |
| Messer et al. | SeeNSearch: A context directed search facilitator for home entertainment devices | |
| US20020059303A1 (en) | Multimedia data management system | |
| Tjondronegoro et al. | Content-based indexing and retrieval using MPEG-7 and X-query in video data management systems | |
| EP1405212B1 (en) | Method and system for indexing and searching timed media information based upon relevance intervals | |
| Tseng et al. | Video personalization and summarization system | |
| Tseng et al. | Personalized video summary using visual semantic annotations and automatic speech transcriptions | |
| US20140250055A1 (en) | Systems and Methods for Associating Metadata With Media Using Metadata Placeholders | |
| US20050033753A1 (en) | System and method for managing transcripts and exhibits | |
| JP3815371B2 (en) | Video-related information generation method and apparatus, video-related information generation program, and storage medium storing video-related information generation program | |
| Khoja | Thematic indexing in video databases | |
| Srinivasan et al. | Multi-modal feature-map: An approach to represent digital video sequences |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment | Owner name:IBM CORPORATION, NEW YORK Free format text:ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ADAMS, HUGH W., JR.;IYENGAR, GIRIDHARAN;LIN, CHING-YUNG;AND OTHERS;REEL/FRAME:013627/0363;SIGNING DATES FROM 20021212 TO 20021216 | |
| STCB | Information on status: application discontinuation | Free format text:ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION |