RELATED APPLICATIONSThis application is a continuation of U.S. patent application Ser. No. 15/057,024, filed Feb. 29, 2016 (Attorney Docket No. IP-P3176US1/EBAY.281118), which claims the benefit of U.S. Provisional Patent Application No. 62/292,753, filed Feb. 8, 2016. Each of the aforementioned applications are hereby incorporated by reference in their entireties.
TECHNICAL FIELDThis application relates generally to data processing and, more specifically, but not by way of limitation, to systems and methods for content-based analysis of media.
BACKGROUNDSome online e-commerce systems (e.g., managing e-commerce sites) allow sellers to offer media such as books or movies for sale. To improve consumer experiences, some online e-commerce systems provide product recommendations to buyers. One known method of generating recommendations for buyers is through “collaborative filtering,” which includes generating product recommendations based on some known interest of a target user (e.g., a product recently purchased by the target user) as compared to known data from other users (e.g., product purchase data from other users that purchased the same product). However, in some situations, there may not be enough data about the product, the target user, or other users for known collaborative filtering methods to perform sufficiently. Further, collaborative filtering performs poorly with cross-category recommendations.
BRIEF DESCRIPTION OF THE DRAWINGSVarious ones of the appended drawings merely illustrate example embodiments of the present disclosure and cannot be considered as limiting its scope.
FIG. 1 is a network diagram depicting an example online e-commerce system.
FIG. 2 is a block diagram showing components provided within the content analysis engine, according to some embodiments.
FIG. 3 illustrates the content analysis engine vectorizing words from a synopsis of a media product (e.g., a book or a movie offered for sale on the online e-commerce system shown inFIG. 1).
FIG. 4 illustrates components of the content analysis engine evaluating a “seed” media product of interest to a user of the online e-commerce system shown inFIG. 1 to provide a recommendation for other media products to the user.
FIG. 5 illustrates a computerized method, in accordance with an example embodiment, for content-based media analysis.
FIG. 6 is a block diagram illustrating an example software architecture, which may be used, in conjunction with various hardware architectures herein described, to perform content-based media analysis (e.g., on the online e-commerce system shown inFIG. 1).
FIG. 7 is a block diagram illustrating components of a machine, according to some example embodiments, configured to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein.
The headings provided herein are merely for convenience and do not necessarily affect the scope or meaning of the terms used. Like numbers in the Figures indicate like components.
DETAILED DESCRIPTIONThe description that follows includes systems, methods, techniques, instruction sequences, and computing machine program products that describe illustrative embodiments of the disclosure. Numerous specific details are set forth herein in order to provide an understanding of various embodiments of the present subject matter. It will be evident, however, to those skilled in the art, that embodiments of the present subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.
A content analysis engine is described herein for providing content-based analysis of media products, such as books or movies offered for sale in an online e-commerce system. In some scenarios, the online e-commerce system has sparse or non-existent purchase data for some media products. For example, a newly released movie or an uncommon (e.g., low-selling) book may have generated little or no prior purchases through the online e-commerce system and, as a result, there may not be enough purchase history for the media product sufficient to generate a satisfactory product recommendation using other methods reliant on such data. To avoid problems with scarce purchase data, the content analysis engine and methods described herein provide content-based analysis and recommendations for media products.
In various example embodiments, the media products offered for sale by the online e-commerce system, and analyzed by the content analysis engine, include a synopsis (e.g., a product synopsis stored as metadata associated with the media product). The synopsis of a book, for example, may include summary text describing the subject matter of the book. The content analysis engine performs content analysis of the media product using the synopsis. More specifically, in one example embodiment, the content analysis engine performs vectorization of each of the words of the synopsis using word embedding techniques (e.g., neural language models for semantic embedding of words in vector spaces, such as with word2vec). Each word of the synopsis is converted into its own word vector. The content analysis engine then combines the word vectors for the synopsis words to generate a media vector (e.g., a combined vector for the synopsis). Accordingly, the media vector is a vectorized representation of the content of the book, via the content of the synopsis.
Similarly, the content analysis engine generates media vectors for each media item in a pool of media (e.g., a catalog of books of the online e-commerce system). Once media vectors have been created for the pool of media, the content analysis engine uses the media vectors to generate one or more media recommendations for users of the online e-commerce system. For example, presume a purchaser buys a book of interest (“seed media”) through the online e-commerce system. The book of interest has an associated media vector (a “seed media vector”), computed as described above. The content analysis engine compares the seed media vector to the pool of media vectors to determine books similar to the book of interest. More specifically, the content analysis engine finds a set of nearest neighbors to the seed media based on the media vectors.
In some embodiments, once the pool of nearest neighbors has been identified, the pool of nearest neighbors is filtered by clustering the seed media and the set of nearest neighbors into a number of clusters. The media products to use as recommendations are selected based on the clusters. In some embodiments, no media item is selected from the cluster containing the seed media product. Some products may be so similar to the seed media product that they are undesirable as a recommendation to the user. For example, a particular book may have multiple versions, and a purchaser of one version is very unlikely to be interested in purchasing another version. Since the two versions of the book are very similar in content, their synopses may also be very similar or identical and, as such, may end up in the same cluster with the seed media product. Thus, providing recommendations from other clusters and not the cluster containing the seed media product reduces the likelihood of recommending undesired content to the user. Further, in some embodiments, only a predetermined number of media products are selected from each cluster (e.g., one media product from each cluster). Limiting the number of media products from each cluster reduces the likelihood that multiple very similar products are recommended to the user (e.g., two different versions of the same book).
In some embodiments, the content analysis engine may also cross-compare types of media. Since the content analysis engine provides a content-based approach, media of disparate types may be compared and recommended based on their synopses. For example, the content analysis engine may compute media vectors of both books and motion pictures. Subsequently, a purchaser may purchase a media product of one type (e.g., a movie about World War II) and the content analysis engine may compare that movie's vector to a pool of book vectors to generate book recommendations that focus on World War II. As such, this content-based approach can cross-recommend media to the user based on the user's interests.
FIG. 1 is a network diagram depicting an exampleonline e-commerce system100. In the example embodiment, theonline e-commerce system100 includes a networkedsystem102 that provides online services to online users, such as auser106 via aclient device110. Thenetworked system102 includes acontent analysis engine150 for performing content-based analysis of media products, and other associated operations, as described herein.
Thenetworked system102 provides network-based, server-side functionality, via a network104 (e.g., the Internet or a Wide Area Network (WAN)), to theclient devices110 that may be used, for example, by sellers or buyers (not separately shown) of products and services offered for sale through a publication system142 (e.g., through an online marketplace system provided by the publication systems142 or payment systems144).FIG. 1 further illustrates, for example, one or more of a web client112 (e.g., a web browser), client application(s)114, and aprogrammatic client116 executing on theclient device110.
Each of theclient devices110 comprises a computing device that includes at least a display and communication capabilities with thenetwork104 to access thenetworked system102. Theclient device110 includes devices such as, but not limited to, work stations, computers, general purpose computers, Internet appliances, hand-held devices, wireless devices, portable devices, wearable computers, cellular or mobile phones, portable digital assistants (PDAs), smart phones, tablets, ultrabooks, netbooks, laptops, desktops, multi-processor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, network PCs, mini-computers, and the like. Each of theclient devices110 connects with thenetwork104 via a wired or wireless connection. For example, one or more portions of thenetwork104 may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a WAN, a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a cellular telephone network, a wireless network, a WiFi network, a WiMax network, another type of network, or a combination of two or more such networks.
Each of theclient devices110 includes one or more client applications (also referred to as “apps”)114 such as, but not limited to, a web browser, a messaging application, an electronic mail (email) application, an e-commerce site application (also referred to as a marketplace application), and the like. In some embodiments, if the e-commerce site application is included in a given one of theclient devices110, then this application is configured to locally provide the user interface and at least some of the functionalities of an e-commerce site, with the application configured to communicate with thenetworked system102, on an as-needed basis, for data or processing capabilities not locally available (e.g., such as access to a database of items available for sale, to authenticate a user, or to verify a method of payment). Conversely, if the e-commerce site application is not included in a given one of theclient devices110, the given one of theclient devices110 may use itsweb client112 to access the e-commerce site (or a variant thereof) hosted on thenetworked system102. Although only oneclient device110 is shown inFIG. 1, two ormore client devices110 may be included in theonline e-commerce system100.
An Application Program Interface (API)server120 and aweb server122 are coupled to, and provide programmatic and web interfaces respectively to, one ormore application servers140. In the example embodiment, theapplication servers140 host thecontent analysis engine150 that facilitates providing analysis and recommendation services, as described herein. Theapplication servers140 are, in turn, shown to be coupled to one ormore database servers124 that facilitate access to one ormore databases126.
In some embodiments, theapplication servers140 host one or more publication systems142 andpayment systems144. The publication system142 may provide a number of e-commerce functions and services to users that access thenetworked system102 or external sites (e.g., a thirdparty publication system130 executing a site application132). E-commerce functions and services may include a number of publisher functions and services (e.g., search, listing, content viewing, payment, etc.). For example, the publication system142 may provide a number of services and functions to users for listing or submitting offers for goods or services for sale, searching for goods and services, facilitating transactions, and reviewing and providing feedback about transactions and associated users. Additionally, the publication system142 may track and store data and metadata relating to listings, transactions, and user interactions. In some embodiments, the publication system142 may publish or otherwise provide access to content items stored in theapplication servers140 or thedatabases126 accessible to theapplication servers140 or thedatabase servers124. Thepayment system144 may likewise provide a number of payment services and functions to users. Thepayment system144 may allow users to accumulate value (e.g., in a commercial currency, such as the U.S. dollar, or a proprietary currency, such as “points”) in accounts, and then later to redeem the accumulated value for products or items (e.g., goods or services) that are made available via the publication system142. While the publication system142 and thepayment system144 are shown inFIG. 1 to both form part of thenetworked system102, it will be appreciated that, in alternative embodiments, thepayment system144 may form part of a payment service that is separate and distinct from thenetworked system102. In other embodiments, thepayment system144 may be omitted from theonline e-commerce system100. In some embodiments, at least a portion of the publication system142 may be provided on theclient devices110.
Further, while theonline e-commerce system100 shown inFIG. 1 employs a client-server architecture, some example embodiments of the present disclosure are not limited to such an architecture, and may equally well find application in, for example, a distributed or peer-to-peer architecture system. The various publication andpayment systems142 and144 may also be implemented as standalone software programs, which do not necessarily have networking capabilities.
Theclient devices110 access the various publication andpayment systems142 and144 via the web interface supported by theweb server122. Similarly, theprogrammatic client116 accesses the various services and functions provided by the publication andpayment systems142 and144 via the programmatic interface provided by theAPI server120. Theprogrammatic client116 may, for example, be a seller application (e.g., the TurboLister application developed by eBay Inc., of San Jose, Calif.) to enable sellers to author and manage listings on thenetworked system102 in an offline manner, and to perform batch-mode communications between theprogrammatic client116 and thenetworked system102.
In the example embodiment, thecontent analysis engine150 analyzes products associated with listings on thenetworked system102. Theonline e-commerce system100 may provide product or listing recommendations to users based on user interest, such as determined through online viewing, watching, or purchasing of products through theonline e-commerce system100.
FIG. 2 is a block diagram showing components provided within thecontent analysis engine150 according to some embodiments. Thecontent analysis engine150 may be hosted on dedicated or shared server machines (not shown) that are communicatively coupled to enable communications between the server machines. In some embodiments, components of thecontent analysis engine150 may be executed on a graphics processing unit (GPU) such as, for example, one or more Titan X GPUs (such as those made commercially available by NVIDIA corporation, a California corporation, and other manufacturers). The components themselves are communicatively coupled (e.g., via appropriate interfaces) to each other and to various data sources, so as to allow information to be passed between the components or so as to allow the components to share and access common data. Furthermore, the components may access the one ormore databases126 via the database servers124 (both shown inFIG. 1).
Thecontent analysis engine150 provides a number of content analysis features related to media products or listings, whereby thecontent analysis engine150 performs content-based analysis of media products to provide recommendations to users. To this end, thecontent analysis engine150 includes amedia vectorization module210, avector combination module220, aneighbor identification module230, aclustering module240, and amedia recommendation module250.
In the example embodiment, themedia vectorization module210 performs vectorization on a synopsis of a subject media product. Themedia vectorization module210 performs vectorization of each of the words of the synopsis using word embedding techniques (e.g., neural language models for semantic embedding of words in vector spaces, such as with word2vec). Each word of the synopsis is converted into a word vector for that word, where each element of the word vector is a real number associated with a dictionary word of the vector. Thevector combination module220 combines the word vectors generated by themedia vectorization module210 for the synopsis words, thereby creating a media vector for the subject media product. As such, the media vector is a vectorized representation of the content of the media, via the content of the synopsis. Thecontent analysis engine150 may perform this vectorization for a catalog of products or listings associated with theonline e-commerce system100, and may store the media vectors with the product catalog (e.g., in the database126).
Thecontent analysis engine150 analyzes a seed media product (e.g., a book or a movie) for similar media products. For example, a user may indicate interest in the seed media product through viewing, watching, or purchasing the seed media product. The seed media product includes a synopsis (e.g., stored in the database126), for which there is an associated media vector (e.g., computed by the vector combination module220). Theneighbor identification module230 examines the seed media product for other similar media products by comparing the seed media vector to media vectors from the product catalog. Theneighbor identification module230 finds a number of “nearest neighbors” (e.g., using k-NN, or k-Nearest Neighbors). In one example embodiment, theneighbor identification module230 identifies R media products as nearest neighbors. Theclustering module240 then clusters the R media products and the seed media product into K clusters, where K<R. As such, most or all of the R media products are placed into one of the K clusters, including the seed media product. Themedia recommendation module250 selects one or more media products from one or more of the K clusters. In some embodiments, the cluster having the seed media product is excluded from generating any recommendations (e.g., no media products in the cluster with the seed media product are recommended). In some embodiments, only one media product from a given cluster is provided as a recommendation.
FIG. 3 illustrates thecontent analysis engine150vectorizing words312 from asynopsis310 of a media product302 (e.g., a book or a movie offered for sale on the online e-commerce system100). In the example embodiment, thecontent analysis engine150 generates amedia vector330 for eachmedia product302 in a catalog of media products (not separately identified) and stores those media vectors330 (e.g., as apool340 of media vectors342) for later use in generating media recommendations.FIG. 3 shows the vectorization process for asingle media product302.
More specifically, themedia product302 includes thesynopsis310, stored in thedatabase126, that describes, in thewords312, a summary of the subject matter of themedia product302. For example, consider a book media product titled “Adrift: Seventy-Six Days Lost at Sea,” by Steven Callahan (2002, Paperback). The synopsis for this book is:
- BeforeThe Perfect Storm, beforeIn the Heart of the Sea, Steven Callahan's dramatic tale of survival at sea was on theNew York Timesbestseller list for more than thirty-six weeks. In some ways the model for the new wave of adventure books,Adriftis an undeniable seafaring classic, a riveting firsthand account by the only man known to have survived more than a month alone at sea, fighting for his life in an inflatable raft after his small sloop capsized only six days out. “Utterly absorbing” (Newsweek),Adriftis a must-have for any adventure library.
As such, thesynopsis310 includes approximately one hundredwords312.
Themedia vectorization module210 vectorizes eachword312 of thesynopsis310 using semantic embeddings of words in a vector space. In the example embodiment, eachword312 is vectorized using “word2vec,” a group of related models and related tools developed and promulgated by Tomas Mikolov and colleagues at Google Inc. (a California corporation). Themedia vectorization module210 uses word2vec to generate aword vector320 for each word (or “user-specified word”)312 of thesynopsis310. Eachword vector320 is a vector of elements, where each element includes a real number value. Word2vec defines the dictionary used, as well as which vector element corresponds to which word of the dictionary. Word2vec is a mapping function. A word that exists in the dictionary is mapped into a vector (e.g., the word vector320) with a fixed size (e.g., 300 real numbers). For example, consider the words “dog”, “cat”, and “pen”. Each of these words may be converted into a word vector, and the Euclidean distance between the two vectors may be determined and compared. Because of the similarities between “dog” and “cat” (e.g., because they are both animals), those two vectors will be closer together (e.g., a nearer or smaller Euclidean distance between the associated vectors) than, for example, “dog” and “pen” (e.g., because one is an animal and the other is a writing instrument). In some embodiments, less than all of the words of thesynopsis310 may be used (e.g., removing duplicates, ignoring articles).
Further, in the example embodiment, themedia vectorization module210 uses a pre-trained model published by Tomas Mikolov, which was trained on a part of Google News dataset (˜100 billion words), and which contains 300-dimensional vectors for 3 million words and phrases. In other embodiments, themedia vectorization module210 may train a model for word2vec (e.g., from a corpus of text that includes more up-to-date data, such as data from Wikipedia). In some embodiments, themedia vectorization module210 may perform vectorization of words using GloVe, “Global Vectors for Word Representation,” as developed by Jeffrey Pennington, Richard Socher, and Christopher D. Manning.
In the example embodiment, thevector combination module220 combines theword vectors320 to form themedia vector330 for themedia product302. In one example embodiment, thevector combination module220 uses mean vector pooling to generate themedia vector330. This pooling technique takes a multiset of vectors, X={x1, x2, . . . , xN} E RD(e.g., where each vector xis one of theword vectors320, R is the set of real numbers, and D is the length of the vectors, x) and computes its mean vector, v (e.g., the media vector330), as:
where N is the number ofword vectors320 output from the media vectorization module210 (e.g., from the synopsis310).
In the example embodiment, themedia vector330 is associated with themedia product302 and stored with thepool340 ofmedia vectors342 in thedatabase126.
FIG. 4 illustrates components of thecontent analysis engine150 evaluating a “seed”media product402 of interest to auser404 of theonline e-commerce system100 to provide a recommendation for other media products to theuser404. Theseed media product402 may be similar to themedia product302. In the example embodiment, thecontent analysis engine150 identifies theseed media product402 for which theuser404 has expressed an interest (e.g., using thecomputing device406, through viewing details of theseed media product402 on theonline e-commerce system100, watching a listing associated with theseed media product402, or purchasing theseed media product402 on the online e-commerce system100). Theseed media product402 is associated with a seed media vector410 (e.g., a vector similar to themedia vector330, generated from a synopsis of the seed media product402). In some embodiments, theseed media vector410 may be pre-computed and retrieved from the database126 (e.g., as described above, where theseed media vector410 is similar to the media vector330). In other embodiments, thecontent analysis engine150 may compute theseed media vector410 contemporaneously, and similarly to the media vector330 (e.g., upon identification of theseed media product402, such as at or just after the time of an online e-commerce sale).
Theneighbor identification module230 then identifies the Rnearest neighbors420 to theseed media product402 based on theseed media vector410 and thepool340 of media vectors342 (e.g., from the catalog of media products). In the example embodiment, theneighbor identification module230 performs a brute-force nearest neighbor approach, computing a spatial Cosine distance between the seed media product402 (i.e., the seed media vector410) and each of the media products in the pool340 (i.e., the media vectors342):
where s is the one-dimensionalseed media vector410, t is the one-dimensional media vector342 of acatalog media product302, and s·t is the dot product of vectors s and t.
Once cosine distances are computed for each pair, theneighbor identification module230 identifies the R nearest neighbors to the seed media vector410 (e.g., theR media vectors342 having the lowest cosine distance). In some embodiments, R is a predetermined number, such as fifty.
In some embodiments, themedia recommendation module250 may select one or more of themedia products422 from the Rnearest neighbors420 and make a recommendation for thosemedia products422 to the user404 (e.g., at or after an online sale of theseed media product402 to the user404).
In the example embodiment shown inFIG. 4, once the Rnearest neighbor420media products422 are identified, theclustering module240 performs clustering on thosemedia products422. More specifically, in one example embodiment, theclustering module240 performs k-means clustering to partition the Rnearest neighbors420, and optionally also theseed media vector410, into K clusters (not separately depicted), where K<R. As such, each of the K clusters contains one ormore media vectors342,410. In some embodiments, themedia recommendation module250 selects onemedia product432 from each cluster (e.g., randomly, or themedia product432 having the closest cosine distance to the seed media product402). Selecting only onemedia product432 from each of the K clusters may reduce the chances of providing recommendations for the same or very similar products to theuser404. For example, clustering enables thecontent analysis engine150 to possibly avoid recommending two different versions of the same book, as the synopses of the two versions may be similar or identical and, as such, they are likely to end up in the same cluster. In some embodiments, themedia product432 selected from each cluster is determined as themedia product432 that was listed the most in the last year. As such, themedia product432 is more likely to have inventory on theonline e-commerce system100. In some embodiments, themedia recommendation module250 may exclude the cluster that includes theseed media product402. Similarly, and for example, excluding the cluster in which theseed media product402 occurs enables thecontent analysis engine150 to possibly avoid recommending a different version of theseed media product402 to theuser404, as the two versions are likely to end up in the same cluster.
In some embodiments, themedia products422 or432 are pre-computed by thecontent analysis engine150 for each seed media product402 (e.g., “catalog book”) in a catalog of media products (e.g., via offline or batch processing). In other words, the recommendations for eachseed media product402 may be predetermined such that, at the time aseed media product402 is identified (e.g., by theuser404 purchasing the seed media product402), thecontent analysis engine150 may retrieve the recommendedmedia products422 or432 for thatseed media product402 from thedatabase126. As such, recommendation processing times may be decreased (e.g., only a lookup, rather than real-time processing), thereby improving the operation of the computer.
In some embodiments, the processing steps are performed using distributed processing. Thecontent analysis engine150 may employ MapReduce in order to execute the feature extraction and pooling steps (e.g., as shown and described relative toFIG. 3). The descriptions of themedia products302 may be stored in a Hadoop distributed file system (HDFS) as a table of (product-id, description). A Hadoop job is splitting the description of each product into tokens of unigrams, bigrams, and trigrams, and storing the results in a table, productToToken, of (product-id, token). The tokens and embeddings of the word2vec model are also stored in a table, tokenToVec, of (token, tokenVec). Storing the tokens and their embeddings in a table instead of in memory allows the Hadoop job to run using many simple machines with a low amount of memory. Another Hadoop job joins the productToToken table with the tokenToVec table, resulting in a new table, productToTokenVec, of (product-id, tokenVec). Another Hadoop job is implementing the pooling and converting the productToTokenVec table into a table, productToRepresentation of (product-id, representation). The GPU brute force nearest neighbor search reads the productToRepresentation table, performs the exact search, and outputs a table, productToCandidates, of (product-id, candidates-products), where the candidates-products field contains a list of the M exact nearest neighbors of the product-id. The diversification step is implemented by another Hadoop job that reads both the productToCandidates and productToRepresentation tables and creates a table, productToRecommendations, of (product-id, recommendations), where the recommendations field contains the final K recommendations that will be presented to theuser404.
FIG. 5 illustrates acomputerized method500, in accordance with an example embodiment, for content-based media analysis. Thecomputerized method500 is performed by a computing device comprising at least one processor and a memory. In the example embodiment, atoperation510, thecomputerized method500 includes generating a media vector for each catalog book of a plurality of catalog books based on an associated synopsis, each catalog book including the associated synopsis. For each catalog book, theoperation510 includesoperation512 for generating a word vector for each word of a plurality of words in the associated synopsis, thereby generating a plurality of word vectors,operation514 for combining the plurality of word vectors into a mean vector, the mean vector being the media vector, andoperation516 for storing the mean vector, in the memory, as the media vector associated with the catalog book. In some embodiments, generating the word vector for each word of the plurality of words in the associated synopsis atoperation512 comprises generating the word vector using word2vec. In some embodiments, the associated synopsis includes N words, the plurality of word vectors includes a multiset of word vectors, X, of N word vectors {x1, x2, . . . , xN}, and combining the plurality of word vectors into a mean vector, v, atoperation514 includes computing:
Atoperation520, themethod500 also includes identifying a target book from the plurality of catalog books, the target book associated with a seed media vector. Atoperation530, themethod500 further includes determining R nearest neighbors for the target book from the plurality of catalog books based on the seed media vector and the media vectors associated with the plurality of catalog books. In some embodiments, themethod500 further includes generating a media vector for each catalog movie of a plurality of catalog movies based on an associated synopsis of each catalog movie, wherein determining the R nearest neighbors for the target book further includes identifying the R nearest neighbors from the plurality of catalog movies, further based on the media vectors associated with the plurality of catalog movies.
Atoperation540, themethod500 also includes clustering the R nearest neighbors for the target book into K clusters. Atoperation550, themethod500 further includes selecting a plurality of catalog books from the K clusters for recommendation to a user. In some embodiments, themethod500 further includes clustering the seed media vector into a seed cluster, the seed cluster being one of the K clusters, wherein selecting the plurality of catalog books from the K clusters excludes the seed cluster.
In some embodiments, themethod500 also includes computing a cosine distance, cosine(s,t), between each pairing of the seed media vector, s, and the associated media vector, t, of each catalog book of the plurality of catalog books:
wherein s·t is the dot product of vectors s and t, thereby generating an associated cosine distance for each catalog book of the plurality of catalog books, and determining the R nearest neighbors for the target book from the plurality of catalog books further based on the associated cosine distance for each catalog book of the plurality of catalog books. In some embodiments, themethod500 also includes selecting one catalog book from each of the K clusters.
Modules, Components, and LogicCertain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules (e.g., code embodied on a machine-readable medium or in a transmission signal) or hardware modules. A “hardware module” is a tangible unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware module that operates to perform certain operations as described herein.
In some embodiments, a hardware module may be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform certain operations. For example, a hardware module may be a special-purpose processor, such as a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware module may include software encompassed within a general-purpose processor or other programmable processor. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software) may be driven by cost and time considerations.
Accordingly, the phrase “hardware module” should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. As used herein, “hardware-implemented module” refers to a hardware module. Considering embodiments in which hardware modules are temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where a hardware module comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware modules) at different times. Software may accordingly configure a particular processor or processors, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.
Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple hardware modules exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware modules. In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information).
The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented module” refers to a hardware module implemented using one or more processors.
Similarly, the methods described herein may be at least partially processor-implemented, with a particular processor or processors being an example of hardware. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an application program interface (API)).
The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processors or processor-implemented modules may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other example embodiments, the processors or processor-implemented modules may be distributed across a number of geographic locations.
Software ArchitectureFIG. 6 is a block diagram600 illustrating anexample software architecture602, which may be used, in conjunction with various hardware architectures herein described, to perform content-based media analysis (e.g., on the online e-commerce system100). Acontent analysis engine680, which is shown in a laywer ofapplications620, may be similar to thecontent analysis engine150, but may be provided in whole or in part at other layers shown inFIG. 6.FIG. 6 is a non-limiting example of asoftware architecture602, and it will be appreciated that many other architectures may be implemented to facilitate the functionality described herein. Thesoftware architecture602 may execute on hardware such as amachine700 ofFIG. 7 that includes, among other things,processors710,memory730, and input/output (I/O)components750. Arepresentative hardware layer604 is illustrated and can represent, for example, themachine700 ofFIG. 7. Therepresentative hardware layer604 includes aprocessing unit606 having associatedexecutable instructions608. Theexecutable instructions608 represent the executable instructions of thesoftware architecture602, including implementation of the methods, modules, and so forth described herein. Thehardware layer604 also includes memory/storage610, which also includes theexecutable instructions608. Thehardware layer604 may also compriseother hardware612.
In the example architecture ofFIG. 6, thesoftware architecture602 may be conceptualized as a stack of layers where each layer provides particular functionality. For example, thesoftware architecture602 may include layers such as anoperating system614,libraries616, frameworks ormiddleware618,applications620, and apresentation layer644. Operationally, theapplications620 and/or other components within the layers may invoke application programming interface (API) calls624 through the software stack and receive a response asmessages626. The layers illustrated are representative in nature and not allsoftware architectures602 have all layers. For example, some mobile or specialpurpose operating systems614 may not provide the frameworks/middleware618, while others may provide such a layer.Other software architectures602 may include additional or different layers.
Theoperating system614 may manage hardware resources and provide common services. Theoperating system614 may include, for example, akernel628,services630, anddrivers632. Thekernel628 may act as an abstraction layer between the hardware and the other software layers. For example, thekernel628 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, and so on. Theservices630 may provide other common services for the other software layers. Thedrivers632 may be responsible for controlling or interfacing with the underlying hardware. For instance, thedrivers632 may include display drivers, camera drivers, Bluetooth® drivers, flash memory drivers, serial communication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi® drivers, audio drivers, power management drivers, and so forth depending on the hardware configuration.
Thelibraries616 may provide a common infrastructure that may be used by theapplications620 and/or other components and/or layers. Thelibraries616 typically provide functionality that allows other software modules to perform tasks in an easier fashion than by interfacing directly with theunderlying operating system614 functionality (e.g.,kernel628,services630, and/or drivers632). Thelibraries616 may include system libraries634 (e.g., C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, thelibraries616 may includeAPI libraries636 such as media libraries (e.g., libraries to support presentation and manipulation of various media formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG), graphics libraries (e.g., an OpenGL framework that may be used to render 2D and 3D graphic content on a display), database libraries (e.g., SQLite that may provide various relational database functions), web libraries (e.g., WebKit that may provide web browsing functionality), and the like. Thelibraries616 may also include a wide variety ofother libraries638 to provide many other APIs to theapplications620 and other software components/modules.
The frameworks618 (also sometimes referred to as middleware) provide a higher-level common infrastructure that may be used by theapplications620 and/or other software components/modules. For example, the frameworks/middleware618 may provide various graphic user interface (GUI) functions, high-level resource management, high-level location services, and so forth. The frameworks/middleware618 may provide a broad spectrum of other APIs that may be utilized by theapplications620 and/or other software components/modules, some of which may be specific to aparticular operating system614 or platform.
Theapplications620 include built-inapplications640 and/or third-party applications642. Examples of representative built-inapplications640 may include, but are not limited to, a contacts application, a browser application, a book reader application, a location application, a media application, a messaging application, and/or a game application. The third-party applications642 may include an application developed using the Android™ or iOS™ software development kit (SDK) by an entity other than the vendor of the particular platform, and may be mobile software running on amobile operating system614 such as iOS™, Android™, Windows® Phone, or othermobile operating systems614. The third-party applications642 may invoke the API calls624 provided by the mobile operating system, such as theoperating system614, to facilitate functionality described herein.
Theapplications620 may use built-in operating system functions (e.g.,kernel628,services630, and/or drivers632),libraries616, or frameworks/middleware618 to create user interfaces to interact with users of the system. Alternatively, or additionally, in some systems, interactions with a user may occur through a presentation layer, such as thepresentation layer644. In these systems, the application/module “logic” can be separated from the aspects of the application/module that interact with a user.
Somesoftware architectures602 use virtual machines. In the example ofFIG. 6, this is illustrated by avirtual machine648. Thevirtual machine648 creates a software environment where applications/modules can execute as if they were executing on a hardware machine (such as themachine700 ofFIG. 7, for example). Thevirtual machine648 is hosted by a host operating system (e.g., operating system614) and typically, although not always, has avirtual machine monitor646, which manages the operation of thevirtual machine648 as well as the interface with the host operating system (i.e., operating system614). A software architecture executes within thevirtual machine648, such as an operating system (OS)650,libraries652,frameworks654,applications656, and/or apresentation layer658. These layers of software architecture executing within thevirtual machine648 can be the same as corresponding layers previously described or may be different.
FIG. 7 is a block diagram illustrating components of amachine700, according to some example embodiments, configured to readinstructions716 from a machine-readable medium738 (e.g., a machine-readable storage medium) and perform any one or more of the methodologies discussed herein. Specifically,FIG. 7 shows a diagrammatic representation of themachine700 in the example form of a computer system, within which instructions716 (e.g., software, a program, an application, an applet, an app, or other executable code) for causing themachine700 to perform any one or more of the methodologies discussed herein may be executed. As such, theinstructions716 may be used to implement modules or components described herein. Theinstructions716 transform the general,non-programmed machine700 into a particular machine programmed to carry out the described and illustrated functions in the manner described. In alternative embodiments, themachine700 operates as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, themachine700 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Themachine700 may comprise, but not be limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a personal digital assistant (PDA), a cellular telephone, a smart phone, a mobile device, or any machine capable of executing theinstructions716, sequentially or otherwise, that specify actions to be taken by themachine700. Further, while only asingle machine700 is illustrated, the term “machine” shall also be taken to include a collection ofmachines700 that individually or jointly execute theinstructions716 to perform any one or more of the methodologies discussed herein.
Themachine700 may includeprocessors710,memory730, and input/output (I/O)components750, which may be configured to communicate with each other such as via a bus702. In an example embodiment, the processors710 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a GPU, a digital signal processor (DSP), an ASIC, a radio-frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, aprocessor712 and aprocessor714 that may execute theinstructions716. The term “processor” is intended to include amulti-core processor712 that may comprise two or moreindependent processors712,714 (sometimes referred to as “cores”) that may execute theinstructions716 contemporaneously. AlthoughFIG. 7 showsmultiple processors712,714, themachine700 may include asingle processor712 with a single core, asingle processor712 with multiple cores (e.g., a multi-core processor),multiple processors712,714 with a single core,multiple processors712,714 with multiples cores, or any combination thereof.
The memory/storage730 may include a memory, such as amain memory732, astatic memory734, or other memory, and astorage unit736, both accessible to theprocessors710 such as via the bus702. Thestorage unit736 andmemory732,734 store theinstructions716 embodying any one or more of the methodologies or functions described herein. Theinstructions716 may also reside, completely or partially, within thememory732,734, within thestorage unit736, within at least one of the processors710 (e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by themachine700. Accordingly, thememory732,734, thestorage unit736, and the memory of theprocessors710 are examples of machine-readable media738.
As used herein, “machine-readable medium” means a device able to store theinstructions716 and data temporarily or permanently and may include, but is not limited to, random-access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., erasable programmable read-only memory (EEPROM)), and/or any suitable combination thereof. The term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) able to store theinstructions716. The term “machine-readable medium” shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions (e.g., instructions716) for execution by a machine (e.g., machine700), such that theinstructions716, when executed by one or more processors of the machine700 (e.g., processors710), cause themachine700 to perform any one or more of the methodologies described herein. Accordingly, a “machine-readable medium” refers to a single storage apparatus or device, as well as “cloud-based” storage systems or storage networks that include multiple storage apparatus or devices. The term “machine-readable medium” excludes signals per se.
The input/output (I/O)components750 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components750 that are included in aparticular machine700 will depend on the type of machine. For example,portable machines700 such as mobile phones will likely include a touch input device or other such input mechanisms, while aheadless server machine700 will likely not include such a touch input device. It will be appreciated that the I/O components750 may include many other components that are not shown inFIG. 7. The I/O components750 are grouped according to functionality merely for simplifying the following discussion and the grouping is in no way limiting. In various example embodiments, the I/O components750 may includeoutput components752 andinput components754. Theoutput components752 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. Theinput components754 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), tactile input components (e.g., a physical button, a touch screen that provides location and/or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.
Communication may be implemented using a wide variety of technologies. The I/O components750 may includecommunication components764 operable to couple themachine700 to anetwork780 ordevices770 via acoupling782 and acoupling772 respectively. For example, thecommunication components764 may include a network interface component or other suitable device to interface with thenetwork780. In further examples, thecommunication components764 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components to provide communication via other modalities. Thedevices770 may be anothermachine700 or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).
Term UsageThroughout this specification, plural instances may implement components, operations, or structures described as a single instance. Although individual operations of one or more methods are illustrated and described as separate operations, one or more of the individual operations may be performed concurrently, and nothing requires that the operations be performed in the order illustrated. Structures and functionality presented as separate components in example configurations may be implemented as a combined structure or component. Similarly, structures and functionality presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.
The embodiments illustrated herein are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. The Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.
As used herein, the term “or” may be construed in either an inclusive or exclusive sense. Moreover, plural instances may be provided for resources, operations, or structures described herein as a single instance. Additionally, boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and particular operations are illustrated in a context of specific illustrative configurations. Other allocations of functionality are envisioned and may fall within a scope of various embodiments of the present disclosure. In general, structures and functionality presented as separate resources in the example configurations may be implemented as a combined structure or resource. Similarly, structures and functionality presented as a single resource may be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of embodiments of the present disclosure as represented by the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.