Methods and devices for managing a play- list of digital content
FIELD OF THE INVENTION
The present invention relates to managing a play- list of digital content such as audio and/or videos and/or the like. More specifically, the invention relates to methods of and devices for managing a play-list comprising digital content items associated with at least one feature parameter. Furthermore, the invention relates to a computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to the invention.
BACKGROUND OF THE INVENTION When playing or rendering digital content such as audio and/or videos and/or the like on a system or device such as an mp3-player, a media center, a home entertainment system, a digital audio and/or video player, a PDA, a computer, a laptop, a mobile telephone, etc. play-lists are typically used for organizing and rendering various digital content items. A play-list may comprise further play-lists or groupings e.g. one for each artist, genre, rating, mood, and/or the like. Typically the play- list is traversed by the player in a random, ascending or descending order. The play- list need not be an actual list of descriptive text containing references to the actual digital content items but may also be a list or collection of graphical icons or the like, each icon representing a specific digital content item.
In such play-lists, when a digital content item (forth only referred to as content item) is unavailable for being rendered by the appropriate player or renderer, the play-list simply skips to the next content item in the list until a playable content item is encountered. This skipping causes an additional pause that is noticeable by a user and thus detracts from the user's experience. Furthermore, if, as an example, the play-list comprises 10 audio tracks and 5 of them are missing or unavailable, a user will only get to listen for half of the expected time, which also detracts from the user's experience. A content item may e.g. be unavailable due to being un-renderable, damaged or simply due to not being present or reachable by the player and/or if a DRM (Digital Rights Management) license or another type of license is not available or reachable. The latter situations can arise quite often if content items are distributed over a variety of devices connected in a network. Especially, if certain portable devices (e.g. an MP3-player) is connected to the network e.g. due to it being powered off, not being connected, etc. Furthermore, content items related in some way (e.g. being from the same album or DVD, same artist, same genre, etc.) will typically be stored on the same device. If such a device (or the digital rights) is unreachable and the current play- list contains several related content items, the digital player will not render several content items causing even longer periods of pause and shorter overall play-time.
Patent application US 2005/0021500 Al discloses a system and a method for repairing play-lists where a user's content experience (e.g. music playing experience) is monitored and metadata describing this experience is collected. Meta-data or information related to a song is used for repairing a play- list. Information related to a song is collected but is based on static and explicit external inputs. The collected information identifying a given song comprises information like title, artist name, album name, genre, etc. (often referred to as the well-known ID3 metadata or ID3 tags) and collected information related to an end- user's experience or preference of the particular song. Static inputs including ID3 tag information are used to determine the song identifiers. However, the ID3 tag information can be, and often is, edited by an end-user and thus cannot be used in a reliable way to identify a song uniquely.
Furthermore, valid metadata may be missing for a given song, i.e. the ID3 information may be blank. This happens for example, when a set of personal recordings is part of a play- list file. If a user for example rips songs from a CD and does not enter any metadata and there is no service running doing this, the tracks will all be named in the same way, simply 'track##' with the number used from the order on the CD and the following metadata 'unknown album', 'unknown artist', etc. The metadata can also be in different languages. Additionally, some of the inputs related to the user-experience may not always be available, e.g. if a user has some songs in the play- list that recently has been purchased. In this situation, information related to the user's experience is not readily available and other inputs like play-count, user rating, etc. will not be known. Additionally, the ID3 information may be incomplete or the information may not be unique since duplication of metadata is possible, e.g. 2 songs may be deliberately or unintentionally be assigned the same metadata by a user. The metadata can also be in different languages.
The disclosure of Patent application US 2005/0021500 Al is only related to repairing a play-list due to a content item being moved and thus only facilitates correction of an old path to the content item. OBJECT AND SUMMARY OF THE INVENTION
It is an object of the invention to provide a method (and corresponding device) of managing a play- list that at least alleviates the above situation(s).
This is achieved by a method (and corresponding device) of managing a play- list comprising at least one digital content item, where the at least one digital content item belonging to the play-list is associated with at least one feature parameter, which has been derived based on the actual digital content item to which it is associated, the method comprising: checking the availability for rendering of a given digital content item belonging to the play- list before the given digital content item is rendered, and if the check results in that the given digital content item is unavailable for rendering then obtaining at least one feature parameter associated with the unavailable digital content item, and determining a replacement digital content item by comparing at least one feature parameter of other content items with the at least one feature parameter of the unavailable content item.
In this way, a simple, reliable and efficient way of managing a play- list is obtained. Furthermore, a user will be presented without any detractions of user experience since the play-list does not skip rendering the unavailable content item(s). Additionally, the user is also presented with a replacement content item that to a high degree should be very similar (at least as best as possible) to the unavailable content item, especially when the feature parameter(s) are carefully selected or derived. In one embodiment, determining a replacement digital content item comprises searching among a number of feature parameters of a number of digital content items by comparing said feature parameters with the at least one feature parameter associated with the unavailable digital content item to locate at least one feature parameter of a digital content item that best matches the at least one feature parameter associated with the unavailable digital content item according to at least one predetermined criteria, and determining a replacement digital content item as the digital content item being associated with the located at least one feature parameter best matching the at least one feature parameter associated with the unavailable digital content item.
In one embodiment, the method further comprises automatically rendering the replacement digital content item instead of the unavailable digital content item.
In one embodiment, the at least one feature parameter associated with a digital content item is a feature vector comprising a plurality of parameters where at least one feature parameter is derived based on the actual content of the digital content item using a fingerprinting or hash technique so it uniquely identifies the content of the given digital content item.
In this way, a very efficient and robust way of identifying content items are obtained resulting in a very efficient and robust way of managing a play-list. In one embodiment, the at least one feature parameter associated with a given content item comprises at least one parameter selected from the group of: a parameter representing a timbre of the given content item or segments or windows thereof, a parameter representing a tone quality of the given content item or segments or windows thereof, a parameter representing a tone color of the given content item or segments or windows thereof, a parameter representing an audio feature of the given content item or segments or windows thereof, a parameter representing an acoustic parameter of the given content item or segments or windows thereof, a parameter representing a spectral envelope of the given content item or segments or windows thereof, a parameter representing a time envelope of the given content item or segments or windows thereof, one or more parameters representing one or more of attack, interonset interval, decay, sustain, release and/or transients of the given content item or segments or windows thereof, one or more parameters representing one or more of a short- time signal level, an average signal spectrum, a modulation cepstra, an average power spectrum correlation of the given content item or segments or windows thereof, and one or more parameters representing a sound or signal parameter or characteristic usable for characterizing the given the entire content item or segments or windows thereof.
In one embodiment, the feature parameters of a content item is comprised by a feature vector and said predetermined criteria or said determining a replacement digital content item comprises subtracting the feature vector for the unavailable content item and a feature vector for a potential replacement content item resulting in a result vector, normalizing said result vector giving a scalar size representing a distance between the unavailable content item and the potential replacement content item where one or more content items giving a scalar size within a predetermined range is determined as one or more replacement digital content items.
In this way, a very efficient way of determining how well content items match the unavailable content item is provided.
In one embodiment, the replacement digital content item is equal to the unavailable content item but being stored at a location different from the location of the unavailable content item. In this way, the unavailable digital content item is replaced by an exact copy from another location.
In one embodiment, the check is done automatically or is done upon request from a user. In one embodiment, a play-list is processed in order to determine an integer number of closest matching other content items for each content item of the play- list and storing an identifier or reference to each closest matching content items for each content item in the play-list and when an unavailable content item is encountered looking up the closest matching content items for that particular content item. In this way, the determination of closest content item(s) will be quicker but will require a bit more pre-processing and a bit more storage.
Other advantageous embodiments of the methods and devices are defined in the sub-claims and explained in the following.
The embodiments of the devices according to the present invention correspond to the embodiments of the methods according to the present invention and have the same advantages for the same reasons.
Further, the invention also relates to a computer readable medium having stored thereon instructions for causing one or more processing units to execute the method according to the present invention.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 schematically illustrates a relationship between a play-list, content items, a replacement content item and associated feature parameter(s) for each content item;
Figure 2 schematically illustrates a rendering device.
DESCRIPTION OF PREFERRED EMBODIMENTS
Figure 1 schematically illustrates a relationship between a play-list, content items and associated feature parameters. Shown are a number of content items (110, 110', 111) and a play-list (100) comprising a representation of a content item and a reference (101) to each content item (110, 110') being part of the play-list (100). A reference (101) would typically contain a reference, URL or link to an actual location of the content item e.g. a local location or a remote location if the content item is stored on a device connected in a network with the playing or rendering device processing the play-list (100). Each content item (110; 110') of the play-list has associated with it at least one feature parameter (130) where the feature parameter(s) (130) has been derived earlier, as explained in the following, based on the actual content item (110; 110') to which it is associated. In one embodiment the number of feature parameters (130) of each content item (HO; 100') is 20. In alternative embodiments the number may be different (both larger and smaller). The feature parameters (130) of the content items (110, 110', 111) are stored in one or more databases, storage structures or similar structures (120) available for query by the playing or rendering device processing the play-list (100). The database (120) storing the feature parameters may be a different (or the same) as the one storing at least part of the content items and may be stored on a separate device. They may be stored locally in connection with the playing or rendering device or at a network connected remote device. Also shown is a content item that is determined to be a replacement content item (111) as explained in the following. The replacement content item (111) also has at least one feature parameter (130) associated with it. In the shown exemplary situation, the content item that is determined to be a replacement content item (111) is not part of the play-list (100) (but may e.g. be part of another play- list or no play- list). In another situation (not shown), the replacement content item (111) may be a part of the play-list (100) and the play-list (100) would comprise a reference to it.
Before a given content item (110; 110') in the play-list (100) is to be played or rendered, the availability for rendering of the content (110; 110') is checked. This check may e.g. be done just about before the content item (110; 110') is to be rendered (giving sufficient time for replacement) or it can be done once a given play-list is loaded or activated or at a time between.
If the check results in that the given content item (110) is available then the playing or rendering device will proceed as normal and render the content item, e.g. by playing or streaming the content item, at the appropriate moment. A content item may be unavailable e.g. due to being damaged, being moved or deleted without the play-list being updated, an associated DRM license not currently being available or if the user has not purchased the associated license, etc. or due to being located at a remote device not being connected with the rendering device at the present moment or the like. In the exemplary situation shown in Figure 1, a content item (110) is available for rendering while another content item (110') is not as indicated by the broken reference (101) connecting the unavailable content item (HO') with the play-list (100). If the check results in that the given content item (HO') is unavailable for being rendered then the following actions are taken. Firstly, the feature parameter(s) (130) of the unavailable content item (HO') is obtained from the database or storage structure (120) using an identifier of the unavailable content item (HO'). Based on the obtained feature parameter(s) (130) of the unavailable content item (HO'), it is determined which feature parameter(s) (130) of other content items that best matches the feature parameter(s) of the unavailable content item (HO') according to at least one predetermined criteria. The content item (111) having the best matching feature parameter(s) with the feature parameter(s) of the unavailable content item (HO') is then designated as a replacement content item (111) for the unavailable content item (HO'). In one embodiment, the feature parameters of a content item are comprised by a feature vector. In this embodiment, the best matching feature parameters of a given content item are determined by subtracting the feature vector for the unavailable content item (130) and the feature vector for the potential replacement content item (111). After subtraction the resulting vector is normalized to arrive at a scalar distance denoting the distance between two content items. The lesser the distance the more the two content items are determined to be similar to each other. The best matching content item(s) are then determined as replacement content item(s) (111).
In one embodiment, the feature parameter(s) (130) for each content item (110, 110', 111) represent its timbre, tone quality, tone color, etc. and/or other audio features and/or acoustic parameters. Such parameters may be extracted using fingerprinting, a hashing technique, etc. The feature parameter(s) (130) of a content item may e.g. represent one or more of spectral envelope, time envelope, attack, interonset interval, decay, sustain, release, transients, short-time signal level, average signal spectrum, modulation cepstra, average power spectrum correlation, etc. e.g. for the entire content item or segments or windows thereof.
In general any sound or signal parameter or characteristic usable for characterizing a content item and/or parts thereof may be used. In one embodiment, at least some of the used feature parameters are robust towards noise or the like.
Please note, the feature parameter(s) (130) need not be extracted on an exact version of the content item. They (or some of them) may e.g. be derived based on a mono version and/or a version having another sampling frequency of the content item that they are derived for.
In one embodiment, only the best matching content item (111) (or content items if they are equally matching) is returned. Alternatively, a plurality of matching content items (111) is returned being within a range of acceptable value(s) or (e.g. adaptable) threshold (e.g. of a weighted sum) for the feature parameter(s). Such a threshold may e.g. be a calculated Hamming distance, which is well known in the art. One of this plurality of content items may then be selected as the replacement content item (111) according to other criteria (e.g. user selection, most similar to last rendered content item, having same user rating as the unavailable content item, having highest play-count, having least play-count where a user has not yet explored the content item, of similar duration as the unavailable content item, from the same artist as the unavailable content item, from the same album as the unavailable content item, etc.). In one embodiment, if no matches are within a certain predetermined threshold or fulfils the other criteria then no replacement is made.
A determination of a replacement content item (111) may be made among only the feature parameter(s) of the content items that currently are available. Alternatively, feature parameter(s) of all content items in the database (120) is checked and the availability of the associated content items is checked at a later time. In this embodiment, if the best matching content item is not available then the second best matching content item is selected and so forth.
In an alternative embodiment, a play-list is processed in order to determine a number (e.g. 10 or generally n, n being an integer) of closest matching (e.g. available) other content items for each content item of the play- list. An identifier or reference to each closest matching content items may then be stored, e.g. in database (120) or another database for each content item in the play-list. When an unavailable content item is encountered the closest matching content items for that particular content item is simply looked up. This requires a bit more pre-processing and requires more storage but the determination of closest content item(s) will be quicker. At the appropriate time for rendering of the unavailable content item (HO') the replacement content item (111), i.e. a best matching (available) content item, is then rendered instead of the unavailable content item (HO').
The check may be done automatically by the rendering or playing device or may be done upon request from a user. In this way, a user will be presented without any detractions of user experience since the play-list does not skip rendering due to the unavailable content item(s). Furthermore, the user is also presented with a replacement content item that to a high degree should be very similar (at least as best as possible) to the unavailable content item, especially when the feature parameter(s) are carefully selected or derived. The replacement content item (111) may both be a different content item or it may be the same content item as the unavailable one but stored in a different location.
The feature parameter(s) (130) of a given content item (110, 110', 111) may e.g. be derived using the well-known so-called fingerprinting technique or a hash function used on the content-item. A fingerprint can be regarded as a short summary of an object. Therefore, a fingerprint function should map a given object consisting of a relatively large number of bits to a fingerprint of a relatively more limited number of bits. This enables a more quick comparison between objects instead of comparing the larger number of bits and also makes the comparison more robust. For good fingerprint functions, a severely degraded signal will still yield a similar fingerprint to a fingerprint of an original un-degraded signal. See e.g. patent application WO 2005/050620 Al (Haitsma, Kalker and Oostveen), incorporated herein by reference, for further details of matching a query data object with a candidate object comparing fingerprints of the data objects using a statistical model of the query fingerprint. See e.g. also patent application US 2002/0178410 Al (Haitsma, Kalker, Bagen and Oostveen), incorporated herein by reference, for a method and apparatus for generating and matching fingerprints of multimedia content where it is described on page 4 how two 3 second audio clips are declared similar if the Hamming distance between two derived fingerprint blocks is less than a certain threshold value.
A fingerprint may e.g. be based on extracting a feature-vector, such as the one used in the LikeMusic™ system from an originating audio or video signal.
Figure 2 schematically illustrates a rendering device. Shown is a rendering or playing device (200) such as an mp3-player, a media center, a home entertainment system, a digital audio and/or video player, a PDA, a computer, a laptop, a mobile telephone, etc. comprising a render circuit (202) comprising one or more processors and a memory and/or storage (203) for storing relevant program information and e.g. content items and the database (120 of Figure 1 if this is not stored at another device or system). The rendering device (200) also optionally comprises an input/output circuit (206) e.g. for receiving user input and giving user output e.g. via an optional display unit (204), which may be integrated in the device (200) or externally connected. The rendering device (200) further comprises a communications circuit (201) for communicating information like queries, content items, results of queries, feature parameters, etc. with another device or system. The rendering device (200) functions as described in connection with Figure 1.
In the claims, any reference signs placed between parentheses shall not be constructed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.
The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.