US20130298001A1 - Presenting actionable elements on a device relating to an object - Google Patents

Abstract

A computing device, method and system is described for displaying actionable elements related to a multi-page printed piece. The device includes an input device, a transceiver circuit and a processing circuit. The input device is configured to receive input data which identifies the multi-page printed piece. The transceiver circuit is configured to transmit a request for data based on the received input data and to receive response data. The processing circuit is configured to generate, based on the response data, actionable elements for each of a plurality of different pages in the multi-page printed piece for display, the actionable elements being selectable by a user to interact with the additional content related to the selected actionable element.

Description

CROSS-REFERENCE TO RELATED APPLICATION
• This application claims the benefit of U.S. Provisional Application No. 61/642,750, filed May 4, 2012, which is incorporated herein by reference in its entirety.
BACKGROUND
• This application generally relates to presenting actionable elements related to an object, such as a printed item, on a device. More particularly, this application relates to presenting electronic information related to the contents of a printed piece or item to facilitate use of or supplement the content of the printed piece.
• Examples of printed publications include catalogs, magazines, newspapers, direct mail, retail inserts, signage, brochures, books, manuals, etc. Some embodiments may be used with printed pieces or items that are not publications. Some embodiments may be used with printed pieces such as single-page or single-surface printed pieces (as opposed to multi-page printed publications such as magazines, etc.), point of sale signage, outdoor signage (such as large signs displayed along a roadway), etc. Some printed publications are filled with URLs, phone numbers, email addresses, physical addresses, and other information that is not electronically accessible.
• Computers, including smart phones and other types of electronic devices, are used as sources for information and entertainment. Electronic devices are being used to, for example, manage a user's schedules, display streaming video for entertainment and informational purposes, provide audio entertainment, display web pages, and display electronic books.
• Computers can display printed text and images in increasingly high resolution and can also provide advanced functionality associated with the text and images. The advanced functionality associated with printed text and images can include providing word definitions, providing links to related information, allowing bookmarking of information, etc. For example, a person reading an electronic book can select an unfamiliar word and request a definition of that word. The computer displaying the electronic book can display a definition of the word from a locally stored database or using an external information source, such as the Internet.
SUMMARY
• A computing device, method and system is described for displaying actionable elements related to a multi-page printed piece. The device includes an input device, a transceiver circuit and a processing circuit. The input device is configured to receive input data which identifies the multi-page printed piece. The transceiver circuit is configured to transmit a request for data based on the received input data and to receive response data. The processing circuit is configured to generate, based on the response data, actionable elements for each of a plurality of different pages in the multi-page printed piece for display, the actionable elements being selectable by a user to interact with the additional content related to the selected actionable element.
• A computing system and method for delivering actionable elements related to a multi-page printed piece comprises a network interface circuit, a memory and a processing circuit. The memory is configured to store identifying information for a plurality of different multi-page printed pieces and associated actionable elements for each printed piece. The processing circuit is configured to receive via the network interface circuit a request for actionable elements for the multi-page printed piece, the request having input information. The processing circuit is further configured to compare the input information in the request to the identifying information stored in the memory, retrieve the actionable elements for different pages of the multi-page printed piece from the memory based on the comparison, and send the retrieved actionable elements via the network interface circuit.
BRIEF DESCRIPTION OF THE DRAWINGS
• Exemplary embodiments are illustrated in the accompanying drawings in which like reference numerals represent like parts throughout.
• FIG. 1 is an illustration of a print experience, showing the use of an electronic device together with a printed piece, according to one illustrative embodiment;
• FIG. 2 is an illustration of various discovery mechanisms that may be used in different embodiments;
• FIG. 3 is a screen shot of a menu showing a plurality of different printed pieces, according to one illustrative embodiment;
• FIG. 4 is a screen shot of a title screen of one printed piece and an actionable element in the form of a hyperlink, a subscribe button, a search function and an open for content function, according to one illustrative embodiment;
• FIG. 5 is a flow diagram of illustrative actionable elements and illustrative actions that may occur in response to selection of the actionable elements;
• FIG. 6 is a flow diagram of illustrative actionable elements and illustrative actions that may occur in response to selection of the actionable elements;
• FIG. 7 is a flow diagram of a system for processing requests for actionable elements, according to one illustrative embodiment;
• FIG. 8A is a flow diagram of a system illustrating an infrastructure for processing requests for actionable elements, according to one illustrative embodiment;
• FIG. 8B is a block diagram of a computing system for storing and serving actionable elements, according to one illustrative embodiment;
• FIG. 9 is an illustration of a data structure stored in a memory implementing a publication information database, the data structure configured to include an electronic document related to a printed publication and associated content, according to an illustrative embodiment;
• FIG. 10 is a flowchart of a method for processing actionable elements related to a multi-page printed piece, according to one illustrative embodiment;
• FIG. 11 is a flowchart of a method for processing actionable elements related to a printed item or other physical object, according to one illustrative embodiment;
• FIG. 12 is a flowchart of a method for processing a request for actionable elements, according to one illustrative embodiment;
• FIG. 13 is a flowchart of a method for building an infrastructure of actionable elements related to different printed items or physical objects, according to one illustrative embodiment;
• FIG. 14 is a flowchart of a method for processing a request for supplemental content, according to one illustrative embodiment;
• FIG. 15 is a flowchart illustrating a method for generating digital elements associated with a printed piece or physical object based on received variant information, according to one illustrative embodiment;
• FIG. 16 is a flowchart illustrating a method for generating digital elements associated with a printed piece or physical object based upon variant information, according to an illustrative embodiment.
• FIG. 17 is a computer implemented augmentation system for presenting digital information related to the content of a printed piece or an object based on received information identifying the printed piece or object, according to one illustrative embodiment.
• Before the embodiments are described in detail, it is to be understood that the systems and methods are not limited in their applications to the details of construction and the arrangement of components set forth in the following description or illustrated in the drawings. These are only illustrative embodiments, and the systems and methods described herein are capable of being implemented in other embodiments and of being practiced in various ways.
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
• Some embodiments herein allow electronic devices to be utilized in combination with printed pieces and physical objects to provide a number of benefits that are not realized by the printed piece or object alone.
• Some embodiments herein provide a more convenient method of obtaining additional information about a multi-page printed piece by using a device to discover the piece once and, in response, receiving additional information and/or actionable elements for one or more than one pages in the piece. In some embodiments, the additional information and/or actionable elements may be obtained without requiring a user to individually discover each portion or page of a multi-page piece.
• Some embodiments allow a user to more easily obtain supplemental content for a printed piece or physical object without having to identify which pages of a piece or portions of an object have actionable elements and which do not, as well as allow the user to engage between the piece or object and the related digital information together
• Some embodiments allow users, such as printers, publishers, advertisers, and others, to make supplemental content for a printed piece or object available without the need for extensive education of end users.
• Some embodiments make the process of building a database of actionable elements for a printed piece or object easier, since fewer discovery mechanisms may be required and fewer steps are required to obtain content.
• Some embodiments may provide less intrusion on pages of a printed piece in the form of QR codes or special imaging features needed for discovery, or the need to have such intrusions on every page or portion of a page where additional information or actionable elements are desired.
• Some embodiments may provide a launch point to provide an end user with a display of or access to supplemental content, actionable elements or additional information for a complete printed piece or a group of related pages or content within a piece, or all pages related to a particular image or launch point, without having to individually take an image of multiple launch points associated with multiple distinct pieces of content.
• Some embodiments may also provide an easier method of producing an interactive experience by using or tying a single launch point to a complete printed piece or group of related pages within a printed piece, such method taking into account versioning and page structure.
• Some embodiments may al low brand owners, advertisers and/or publishers of printed materials and objects to provide hyperlinks to a deeper page of a web site than would be printed in the printed version of the materials or available on the object itself.
• Referring now toFIG. 1, an illustration of a print experience is shown. In this embodiment, a printed piece10 (in this example a printed publication) is shown being read by a user. The same printedpublication10 may be read in print form alongside amobile computing device12 implementing one or more features described herein.
• Printedpublication10 may comprise content (e.g., text, images, editorials, advertisements, graphical content, etc.) printed on a physical, tangible, medium, such as paper, and published to a distribution of recipients, sometimes on a periodical basis. Printedpublication10 may be a multi-page publication. Examples of printed publications include catalogs, magazines, newspapers, direct mail, retail inserts, signage, brochures, books, manuals, etc. Some embodiments may be used with printed pieces or items that are not publications. Some embodiments may be used with printed pieces such as single-page or single-surface printed pieces (as opposed to multi-page printed publications such as magazines, etc.), point of sale signage, indoor or outdoor signage (such as large signs displayed along a roadway), etc.
• For a publication printed in different versions, the versions may be based on one or any combination of segmented data based on characteristics, needs and preferences of the end user of the printed publication.
• Mobile computing device12 may be any of a variety of different types of computing systems that are designed to be used portably. Some are hand-held devices, such as a smartphone or a wrist-wearable computer, since they are configured to be used while being held in a person's hand. Some are not hand-held, but nevertheless mobile, such as laptop computers, wearable glasses, etc. In other embodiments, non-mobile computing systems may be used, such as a desktop computer, television, connected TV, smart TV, monitor, etc.
• Althoughsystem110 is described with reference to a typical computing system, it should be understood that the system and method described herein could be implemented in a variety of ways. One such example could be wearable glasses including an integral display and processor for implementing the functionality described herein.
• In this embodiment,mobile computing device12 is configured to run a computer application that may be downloaded to the device. The computer application can facilitate the functions described herein of discovering printedpublication10, retrieving actionable elements or other supplemental content, displaying actionable content, etc.
• Referring now toFIG. 2, some discovery mechanisms that may be used in different embodiments are illustrated. In some embodiments, a discovery mechanism for a printed item may be used to identify the printed publication using an internet-enabled smartphone or tablet device to prompt the device to display or otherwise provide access to additional web-enabled content, which may comprise audio/visual content, audio content without visual content, haptic feedback, or other content.Mobile computing device12 may operate a computer application configured to provide a publicationidentification home screen14 on a display device to a user having a plurality ofselectable options16 for activating different discovery mechanisms.
• A Quick Response (QR) code or other 2D bar code may be used as a launch point on some publications.Device12 may provide barcode input button18 on the display (or alternatively, an application or other program may be configured to automatically recognize, for example without requiring user input or selection, a bar code or other image element). In response to selection of barcode input button18, a camera ondevice12 may be activated with the image received at the camera shown ondisplay14. Based upon selection of the barcoderecognition function icon18,device12 may be configured to display abarcode recognition screen19 displaying a digital image captured by an integrated camera ofdevice12.Home screen19 may be configured to provide instructions to the user regarding framing of the barcode, to assist in recognition of the barcode. The user may place the camera over aQR code20, anddevice12 may automatically (i.e., without requiring further user input), or in response to user input, acquire an image of the QR code and thereby read the code.
• In another embodiment, a wireless ID tag, such as a Near Field Communication (NFC) device or other Radio Frequency Identification (RFID) device, may be used as part of a discovery mechanism.Device12 may be brought near a printed item close enough to allow the transfer of an ID code todevice12.Device12 may have an integrated read that can continuously scan for the presence of such codes, in a manner that little or no user input is required to accomplish the discovery.
• In another embodiment, a textual search function may be used for discovery. Based on selection of the manualentry function icon20,device12 may be configured to display a searchingscreen22 including a virtual keyboard23 configured to allow a user to manually enter one or more keywords intended to help identify a particular printed publication. The user can provide information about the printed publication and/or content provided therein such as title, article name, date, image information, or other publication or content information.Screen22 may be configured to display a listing of identified printed publications in aninformation section25 based upon keywords entered by the user. In response to selection of a textual search input button oricon20, asearch screen22 may be shown allowing the user to type in a search query to find the printed piece.
• In another embodiment, an image recognition function may be used for discovery. The user may select an image recognition input button oricon24. In response to this selection,device12 may be configured to display on image recognition home screen14 a digital image captured by an integrated camera ofdevice12. Home screen310 may be configured to include instructions to the user for capturing animage using device12. The user captures an image of all or a portion of a page of the printed item, such as a cover of a printed publication, as shown at26.Device12 may use points in the image to identify the printed item, with processing handled either locally ondevice12 or remotely at a server computer.
• As shown, the discovery may be made through interaction with a launch point on the printed publication, such as a QR code, other bar code, NFC tag, or portion of an image, or by a user typing in a search query for the publication, or by a user scanning an image of a shape or sculptural aspect of an object. Other discovery techniques may be implemented.
• A software application installed on the user's device can read the launch point and direct an internet browser or other application installed on the device to a specific URL or augmentation information associated with, or included in the launch point of the printed piece, as will be described in greater detail below. Alternatively, upon reading the launch point, the software application may open or launch other software applications for alternative uses.
• In one embodiment, a single input data or code obtained from a multi-page printed item may be used bydevice12 to obtain actionable elements for a plurality of different pages, without requiring a user to scan each page of the multi-page printed item.
• Content displayed as a result of reading the launch point may relate to an object associated with the launch point. For example, where the displayed content is tied to an advertisement, scanning the launch point may display a web page directed to a product, a brand, an article or other editorial content that is directly associated with the launch point, as will be described in greater detail below.
• Another discovery mechanism can be an algorithm that deduces, predicts, or determines an edition of a printed publication for which a user will want actionable elements. For example, if a user discovers the February edition of a magazine, the system can suspect that the user will want the March edition of the same magazine. Further, the system can automatically submit supplemental data for the March edition to a screen such as that shown inFIG. 3; and this submission may occur prior to the printed version is shipped, or based on information about the known ship date, the actual date that the printed version is scheduled to arrive at the user's home.
• FIG. 3 is a screen shot of a menu showing a plurality of different printed pieces, according to one illustrative embodiment. In response to a textual search query or a code received from a launch point,device12 may be configured to search for publications that match the query. The search may be done locally ondevice12, for example in a list of previously-stored “favorites” or known subscriptions the user has to printed publications, and/or the search may be done on a remote server computer against tens, thousands, or more printed publications.Device12 may generate display data representing a plurality of different multi-page printedpieces30,32 for simultaneous display on a single screen.
• In this example, a user has typed “Milwaukee” into asearch field29.Device12 has searched a remote database of potential matches by title of publication, and two results are shown: “Milwaukee Magazine January 2012”30 and “Milwaukee Magazine October 2012”32. A thumbnail image of a front cover of the magazine may be displayed in an active area of the screen, to allow a user to press the screen to select actionable elements for the selected publication. The thumbnail image may have a smaller file size than a full-size image of the page stored on a remote server. In response to selection of one of the multi-page printed pieces being simultaneously displayed, a plurality of actionable elements for the selected piece may be displayed.
• FIG. 4 is a screen shot of a title screen of one printed piece and an actionable element in the form of a hyperlink, a subscribe button, a search function and an open for content function, according to one illustrative embodiment. In this embodiment, selection of one ofpublications30 and32 (FIG. 3) leads to atitle screen40 illustrating a larger thumbnail image of the printed publication.Title screen40 may provide additional actionable or convenient items, such as asubscribe button42, which can be touched or tapped to take the user to a subscription page configured to sign the user up for a subscription (for example, by taking suitable delivery address, payment information, etc.) Awebsite input button44 may be provided, which can be tapped to take the user to a desired website, for example, using a browser application operating ondevice12.
• Aninput button46 may be tapped to display additional actionable elements. The actionable elements may be downloaded todevice12 prior to the tap or in response to the tap.
• FIG. 5 is a flow diagram of illustrative actionable elements and illustrative actions that may occur in response to selection of the actionable elements.Device12 may be configured to present supplemental publication information using a plurality of publication information screens50-60 is shown. Although screens50-60 are shown by way of example, the display screen can provide the functionality described herein in many alternative ways.
• An actionableelement home screen62 or64 can be displayed ondevice12 based upon the publication ID code or data received as described above with reference toFIGS. 2-4. The information displayed onhome screen62 or64 may be related to information in the physical printed publication10 (FIG. 1). Advantageously,home screens62,64 display a plurality of actionable elements and/or other descriptive information in a related group of content. In this embodiment,home screen62,64 may provide the end user with a display of or access to all actionable items for a complete printed publication, or a group of related pages within a printed publication, or all pages related to a particular image or launch point, without having to individually take an image of every page or barcode or tag.
• Device12 is configured to receive data from local memory or a remote server in response to the ID code discovered for the printed publication. From this received response data (e.g., hyperlinks, text data, phone numbers, etc.),device12 generates actionable elements70-94. In the illustrated embodiment, actionable elements are generated for each of a plurality of different pages in the multi-page printed piece for display onscreens62,64. An actionable element is an element provided bydevice12 which is selectable by a user to retrieve and/or display additional content related to the selected actionable element.
• For example,actionable element70 is a field ofscreen62 comprising a hyperlink to www.bayshoretowncenter.com, an active area of the screen acting as a user input device, and a textual display of the element, namely “Bayshore Town Center.” In response to selection of the actionable element, a mobile web browser is launched to display the web site referenced in the hyperlink, as shown byscreen50.
• As another example,actionable elements74 and76 direct a user to social media applications operating ondevice12 referenced to Bayshore Town Center, for Facebook and Twitter, respectively.
• As another example, an actionable element may be selected by a user to provide an augmented reality experience with respect to a corresponding page in the printed publication. For example, a 3-D image, video, or other content may be overlaid on an image of the printed page, as shown byscreen56. The augmented reality content may be downloaded in response to selection of the actionable element, or may be downloaded prior to selection of the actionable element. The augmented reality content may be anchored to or aligned with the image taken by the mobile device, or it may be not anchored to the image.
• As another example, anactionable element88 may comprise a hyperlink to a mapping application configured to show a map of the location of the retail store referenced in the actionable element, as shown byscreen58.
• As another example, anactionable element90 is a phone call element. In response to selection of this element,device12 launches a phone application and dials the phone number displayed inelement90. The phone call may be implemented with a single touch ofelement90, or with a touch-and-confirm method after displaying the phone number within the phone application.
• As another example, anactionable element92 may comprise a hyperlink or other reference to launch video content related to the page or printed content of the printed publication referenced by theactionable element92, as shown byscreen60.
• A user, usingdevice12, can select any information being displayed on the information screens62,64 to display information detail screens50-60. Information detail screens50-60 may be used to launch web pages, display additional images, display rich media content, display additional information, and implement additional functionality such as mapping, making reservations, etc.
• Screens62 and64 are merely illustrative. In one embodiment, the screen is configured to display actionable elements for a plurality of different pages of a multi-page publication simultaneously on a single screen on the display. The single screen may be scrollable to reveal additional information on the screen. For example, screens62 and64 may be different portions of a single screen.
• The actionable elements may be arranged for display based on page numbers of the multi-page printed piece relating to each of the actionable elements. For example,screen62 shows “p 10” in aheader bear63, indicating that the actionable elements70-76 all relate to content onpage 10 of the physical printed publication corresponding to the actionable elements. In this embodiment, the actionable elements are arranged from top to bottom of the screen in the order that their corresponding printed content appears in the printed publication. This arrangement provides a view in ways like a “table of contents” in that it is arranged by page number. Alternatively, the contents may be arranged topically or in any other logical order, such as chronologically, etc. This arrangement may allow a user to easily locate the actionable elements for pages being viewed by the user in the physical copy of the printed publication (as inFIG. 1).
• FIG. 6 is a flow diagram of illustrative actionable elements and illustrative actions that may occur in response to selection of the actionable elements. Actionable elements102-112 are displayed onscreen100. As one example, in response to selection ofelement104, details of a product shown on the page of the printed publication (e.g., a product sales catalog) may be shown, such as price, customer ratings, pictures, etc. The element may comprise an indicator, such as a letter or number, to assist the user in correlating the actionable element to a product shown in the printed publication at the page referenced by thepage header103. Fromscreen120 or122, or from another actionable element onscreen100, a user may “add to cart” or complete additional steps in the purchasing of the product being viewed, resulting in screen124 (e.g., a shopping cart or payment/shipping screen).
• As another example, selection of a marquee advertisingactionable element101 may bring the user to a product search mobile website page, as shown atscreen126. The user may then search by keyword or product number to obtain a listing of products available from that advertiser, as shown inscreen128.
• As another example, an actionable element may be selected to request a paper catalog, as shown byscreen130. For example, a user may have discovered the publication using a text search without having or owning a printed copy of the catalog. In this case, the user is viewing the actionable elements without the printed copy of the publication in hand. In this scenario, or others, the user may be presented with a “request a catalog” actionable element to direct a user to a screen that will receive the user's address, phone, or other information needed to ship a paper copy of a catalog to the user's physical address.
• As another example, an actionable element may be selected to “find a store”, as shown byscreen132. As another example, an actionable element may be selected to “buy now” a particular product.
• FIG. 7 is a flow diagram of a system for processing requests for actionable elements, according to an illustrative embodiment. On adevice side110,device12 may comprise an input device (such as a camera, text search box, wireless transceiver, microphone, etc.) configured to receive input data which identifies the multi-page printed piece.Device12 may comprise a wired or wireless transceiver circuit (e.g., a Wi-Fi circuit, an Ethernet circuit, a cellular data circuit, etc.) configured to transmit a request for data based on the received input data and to receive response data. A processing circuit withindevice12 is configured to implement the functions described herein.
• On a server side,114, a server computer (e.g., comprising one or more physical computing systems, a cloud computer, etc.) is configured to deliver actionable elements todevice12. Server computer may comprise one or more functional circuit modules (e.g., programmed portions of a processing circuit) and a network interface circuit, which may comprise any suitable electrical circuit and associated components for communicating with a network, such as the Internet.
• Server side may also comprise amemory118, which may comprise one or more databases.Database118 may be a repository of information including electronic documents and other content associated with one or more printed publications. Electronic documents may comprise a complete image of a printed piece or digital elements related to or associated with the printed piece. In an alternative embodiment, one or more portions of the information and/or functionality ofdatabase118 may be stored locally ondevice12.
• Associated content indatabase118 associated with the content in the printed publication can further include information about attributes such as page number, layout, location, orientation, image and/or product information such as SKU, date, digital rights management, author, owner, caption, color, shape, links storyline, and transaction information such as price, size, UPC, shipping and other types of information. Moreover, extra or associated data feeds related to the content may be associated with the content. These data feeds may be in the form of text, image, video, and may be associated with a digital address such as a URL. Web and file links may further be associated with any of the content. The content associated with the data feeds can be promotional or informational.
• Database118 may define a lightweight content structure, which serves as an outline, or a skeleton, for each final print media version, distributed. This database may include all of the print version parameters such as title, date, number of pages, location destination, etc. The content outline is also a container structure from which to digitally distribute print media augmentation or supplemental content (e.g., actionable elements) as well as print reference or look-up information such as images, pages, watermarks, embedded codes, keywords, etc.Database118 can further include information about attributes such as page number, layout, location, orientation, image information such as SKU, date, digital rights management, author, owner, caption, color, links storyline, and transaction information such as price, size, UPC, shipping and other types of information. Moreover, extra or associated data feeds related to the content may be associated with the content. These data feeds may be in the form of text, image, video, and are generally associated with a digital address such as a URL. Web and file links may further be associated with any of the content. The content associated with the data feeds can be promotional or informational. The content and associated information can be stored on a memory in a dedicated server or cloud environment.
• In one embodiment,database118 may contain files suitable for use by a printing press (e.g., a digital laser printing press, a web offset printing press, a gravure printing press, or other presses) for large-scale commercial printing. Such printing presses may have multi-color print towers (e.g., cyan, yellow, magenta, black), plates, blankets, and other features of large-scale commercial printing presses. In such an embodiment,database118 may serve data to both printing presses and to mobile devices. For example, full images may be printed by the printing presses, and something less than full images, such as actionable elements and/or thumbnails having smaller files sizes may be served to mobile devices for use with the printed publication.
• The actionable elements may correspond to printed text in the multi-page printed piece, wherein the actionable elements are displayed in a format different than a format of the printed text. For example, a page of a magazine may have an image of an advertisement comprising a textual portion listing a web site address. The corresponding, related actionable element may comprise the textual portion comprising the web site without the full corresponding image. Advantageously, actionable elements may provide additional information or features that need not be published in the print version of the publication, such as a link to a web page deeper within a web site.
• An infrastructure implemented onserver side114 may provide access to content of a printed publication to an end-user on an end-user device.
• The infrastructure may receive a request in relation to the printed publication viarequest120. Amodule122 may look up the issue of the publication and the actionable elements by searchingdatabase118. The request may be compared with identifying information (e.g., IDs, codes, data, etc.) associated with the content indatabase118.Database118 may compriseadvertisements124, images of pages in the printed publication, and/or metadata indexed by title and/orissue128.
• At amodule130, the infrastructure is configured to process the data fromdatabase118 for transmission todevice12. For example, the actionable elements may be stitched together or otherwise combined into a file, arranged or organized by page number, formatted fordevice12 based on the type of device (e.g., operating system, browser type, device capabilities, display size, wireless data connection type, etc.), and/or other information contained in the request or other otherwise processed. This results in data organized in a framework corresponding to the printed pages of the printed publication, as illustrated atframework132. The processed content associated with the printed publication may be provided to the end-user device viaresponse data134.
• Implementations are not limited to the structures or functions illustrated inFIG. 7. For example,device12 may provide some of the functionality ofmodule130 to format the response data for display as actionable elements. Actionable elements may be generated atdevice12 based on processing steps occurring ondevice12,module130, and/or other portions of the system.
• FIG. 8A is a flow diagram of a system illustrating an infrastructure for processing requests for actionable elements, according to one illustrative embodiment. In this flow diagram, pages of a printedpublication140 are shown. In one example, the digital data files for the pages may be available at a printing operator, and may include .pdfs or other image files of the actual pages to be printed, version data for different versions of the publication, management information system (MIS) information for the publication, time/date of printing, .xml files, and other data.
• The server side computing system may be programmed to create the mobile application infrastructure or framework which may reflect all print work, be structured to support mobile interaction and rich media connections to print, and/or provide content and statistics analytics. Aframework132 comprises data generated by the server side computing system representing the layout by page of the printed publication.Actionable elements142,144,146, etc. may be associated with certain page of the printed publication, based on related information printed on the page.
• FIG. 8B is a block diagram of a computing system for storing and serving actionable elements, according to one illustrative embodiment. A content manager anddatabase system150 is a computing system configured to build an infrastructure of actionable elements related to different printed items.System150 comprises one or more network interface circuits configured to facilitate communication with other computers over networks, such as the Internet.System150 comprises a memory configured to store identifying information (e.g., IDs, codes, data, image data elements for image recognition, etc.) for a plurality of different printed items to be printed for different customers. The identifying information may be any type of data or code configured to uniquely identify a printed item or items. In this illustrative embodiment,system150 may be maintained by a printing company, software services company, or other entity. These entities may serve single customers or a plurality of different customers, such as magazine publishers, packaging manufacturers, advertisers, etc.
• System150 may be configured to receive, for each printed item, a selection of a discovery mechanism for discovering the printed item from a plurality of different discovery mechanisms and at least one actionable element.
• The selections may be input manually into the database by theentity operating system150 on behalf of its customers, as indicated bymanual entry block152. For example, text entry may be used. Alternatively, or in addition, the selections may be input via acustomer interface portal156, which comprises a web site accessible with secure credentials by a customer to input its own data for its own printed items. The selection of a discovery mechanism can be by way of a checklist presented to the user (e.g., “_QR code, _image recognition, and/or _RFID tag”), by allowing the user to graphically select and place a launch point on the printed item, or by other methods.
• System150 may also be configured to receive selections of actionable elements providing access to content or functionality supplemental to content printed on the printed item. The supplemental content may be hyperlinks, augmented reality files, video files, audio files, image files, charts or graphs, links to web pages, etc. The selections of actionable elements may be received viamanual entry152, viacustomer interface portal156, etc.System150 may be configured to associate content with the printed publication, and store the content in a memory.
• In another example, a digitalcontent scraping system154 can be used to scan or read an image file showing the printed item, to identify potentially actionable elements in the image file, and to remove or scrape those items from the image and store them in memory. For example, the pages of a magazine may each be read to identify any URLs, phone numbers, addresses, social media icons, or other recognizable text or images, to provide a list of data items that can be used to generate actionable elements. Optical character recognition, or any other reading or scraping technologies may be used. A customer or other person may review the list of scraped items, edit, and store the edited contents in memory.
• In another example, a publisher may usecustomer interface portal156 to upload a structured spreadsheet document (e.g., Microsoft Excel, etc.) containing hyperlinks or other supplemental content.
• In another example, a publisher may use an advertisement portal, such as an ad portal used to insert advertisements to the print process through a separate data stream than the images for the pages to be printed, to upload supplemental content. Supplemental content may loaded intosystem150 through this portal. In one example, an advertising agency may upload an advertisement to be printed in a plurality of different publications. In this case, the supplemental content can be uploaded once and linked to the multiple different publications. In one embodiment, the supplemental content can be overlaid with different brand creative based on the printed publication that is to carry the printed version of the supplemental content.
• In another example, a web feed format, such as an RSS feed, may be used to update the supplemental content.API161 may provide yet further access todatabase160 by way of one or more application programming interfaces accessible by other entities.
• In another example, geolocation, or ZIP-code and USPS carrier route data can be used to add discovery element links.
• System150 is configured to store the selection of the discovery mechanism and the actionable element or elements in the memory in association with the ID for each of the plurality of different printed items to be printed for different customers.
• Auser activity block158 is a processing circuit or part of a computing system configured to record activity a user performs withdevice12. A user may be given an option of having some of the user's activity recorded, for example by an “opt in” or “opt out” screen displayed ondevice12. If the user does opt to have activity recorded, any user interactions with an actionable element described herein may be recorded. An amount of time a user spends reviewing certain items may also be recorded. A user may store a user profile ondevice12 comprising areas of interest (e.g., golf, swimming, sports, etc.), name, location, home location, work location, whether a user is a standard customer or preferred customer for a particular retailer, customer loyalty codes or numbers, coupon codes, or other information the user feels comfortable sharing withsystem150.
• In one example,system150 may be configured to select a particular actionable element or set of items for a user based at least in part on the user interaction data and/or personalized data for the user. One example is to select a particular version of a set of actionable elements based on a version of a printed publication. Versioning occurs, for example, when similar publications are printed in multiple versions, each version having content specific to a group or person based at least in part on segmented data or personal data. Segmented data generally refers to data dividing groups of individuals into segments that share similar characteristics, needs or preferences. As an example, data may be segmented by demographic or geographic information. Demographic information may refer to the statistical characteristics of a population. Commonly used demographic characteristics may include age, abilities, mobility, home ownership, employment status, income, education level, occupation, marital status, parental status, and other similar types of data that could be used to characterize a population. Geographic information may include information characterized by the physical location of the customer or other data source. Commonly used geographic characteristics may include geopolitical information (e.g., nation, state, region, county, city, or neighborhood), postal information (e.g., zip code-based information), global positioning information, etc. In addition to market segmentation, versions of a publication may be based upon personal data or attributes such as the characteristics of an individual as opposed to a population of people. Characteristics that could be used for personal data or attributes can include name, behavior, age, etc., but also could include a number of characteristics shared with demographic and geographic information.
• System150 may be configured to receive personalized data for a user (e.g., user interaction data, user profile data, segment data, version data, etc.) and to retrieve the actionable elements based further on the personalized data for the user. The personalized data may comprise historical user interface data, the historical user interface data representing previous interactions between a user and a user's mobile computing device. The historical user interface data may comprise user selections of actionable elements displayed on the mobile computing device.
• In one embodiment,system150 may be configured to arrange the actionable elements based on the personalized data for the user. For example, actionable elements in a same or similar category to a user's interest, as stored in a user profile, may be placed first in a list of actionable elements on a page. The selections and arrangements described herein may alternatively be implemented ondevice12.
• In another embodiment, a context aware platform provided by a third party, such as Gimbal's platform, Qualcomm Labs, Inc., may be used to provide personalized data for users that can be used to trigger, generate, update or modify supplemental content or actionable elements.
• In another example,system150 may be configured to make available tocustomer portal156 user interaction data relating to the printed items associated with that customer, for example in the form of a report. This may allow the customer to evaluate the effectiveness of certain advertisements or other content, based on whether a user interacts with the content (e.g., an augmented reality item, hyperlink, etc.), how long or how extensively a user interacts with the content, whether a user makes a purchase of an item using an actionable element, etc. Access may be provided securely to different customers (e.g., via username/password or other credentials) to maintain confidentiality.
• An advertiser/brand owner block160 represents access tosystem150 by an advertiser or brand owner. An advertiser may provide actionable elements or updates to actionable elements stored insystem150, for example, to reflect sales, sale dates, inventory of products, replacement products for products that are sold out, IDs of customers who are preferred customers (e.g., pricing categories for customers or by location or other segment), etc.System150 may be configured to select, edit, or update actionable elements received viablocks152,154 or156 based on this additional information received fromblock160.Block160 may represent a secure web portal accessible by advertisers, brand owners, ad agencies, or other entities.
• Channels162 represent the different channels of media from which users can access the information insystem150 or to which the system can deliver information associated with printed items or objects. Any or all data insystem150 can be used by a printer for printing in-store signage, point of sale signage, or packaging materials, by a press operator for printing publications, by a web site operator for presenting and updating product offerings and content, by a digital publisher for publishing a digital edition of a magazine or other publication, by an outdoor advertising agency for publishing outdoor advertisements, or for use in other media channels. Select data elements can be requested fromsystem150, without the need to download an entire preassembled page with all of its collective links and other supplemental content. This may allow content owners or providers to make content available for use in multiple channels without having to use separate workflows. This may also allow them to create content for different segmented markets without having to use separate workflows.
• The various data feeds intosystem150 may be static or dynamic. Static feeds may include data being entered manually, for example. Dynamic feeds may include data being continuously or periodically updated as it is acquired, typically without requiring manual user input for each data element.
• FIG. 9 is an illustration of a data structure stored in a memory implementing apublication information database118, the data structure configured to include anelectronic document200 related to a printed publication and associated content, according to an illustrative embodiment. The form and content of the database may be varied to customize the functionality described herein.
• Anelectronic document200 may be configured to include a plurality ofsignatures202 and/or digital images204 representative ofsignatures202. In a printed publication, a signature is a large sheet printed with multiple pages that when folded becomes a section of the printedpublication100. Accordingly, each signature may be associated with multiplesignature content sections206 in theelectronic document200.
• Signature content sections206 define a page structure that is the architecture and building blocks of the printedpublication100.Content sections206 may further be subdivided based on the content in any particular signature content section, such as by specific articles, images, advertisements, etc.
• Asignature content section206 can includeinformation208 about the discovery mechanism used by a system to correlate the content in the printed publication with a particularsignature content section206. For example,identification information208 can include a listing of displayed bar codes, an image for image matching, etc.
• Signature content sections206 can further includecontent detail information210 that includes page structure information including the design and layout of individual pages and how those pages are arranged and coordinated in the complete printed publication. Page structure also can include taxonomy and tagging of the printed publications and their associated pages and content. Examples of page structure can include primary information about the printed media itself, such as publisher, title, issue and version information. Page structure also can include page and content information, such as one or more of the following: page number and layout type, whether the page or content relates to editorial or advertisement, whether the page or content is image or text, the orientation and/or location information of the page and content, image information such as SKU, date, digital rights management, author, owner, caption, and color data, text information such as storyline, transactional information including price, volume, size, code, etc. information, any links to web or file destinations, extra or associated data feeds. In one embodiment,structure section206 or portions thereof may be used to tie content information to the particular content in the printedpublication100 in which it is published. Table 1 below provides an exemplary listing of page structure information.

• Printed Media Information	Page and Content Information
  Publisher Information (CRM	Page (number and layout type)
  information such as name,
  address, historical information,
  market type, etc.),
  Title Information (name of
  printed publication)
  Issue (volume no, date, season,	Editorial
  etc.)
  Version Information (demo-	Advertisement
  graphic, geographic, or other
  segmented type)
  	Video Information
  	Image Information
  	location on the page
  	orientation
  	image information (SKU(s) for product
  	on the page, Date product or sale is
  	good for, digital rights management
  	(DRM), Author, Owner, Caption,
  	Color, Links, Ad or Editorial)
  	Text information
  	location
  	orientation
  	description (storyline, transactional
  	information)
  	Extra/Associated data feeds (may add in
  	extra data to images- typically asso-
  	ciated with opt-in experience such as
  	special offers, promos, video, additional
  	info, augment reality events,
  	Web and File Links

• Signature content section206 may be associated with content variants212. A content variant212 is a variant of a digital signature having a variation to at least some of the content associated with thesection206. Content variant212 may include a regional variant214 to customize the content in the content section based on geographic information and/or a segment variant216 to customize the content in the content section based on segment information.
• Electronic documents200 or digital elements associated with the documents may be automatically generated during a printing process for the printedpublication100. Each signature may be initially generated by capturing digital images or .pdfs during the printing process to form thesignature202 and content section204. Further, as different versions of a printed publication are printed, image processing software can be configured to detect variations for inclusion in content variants212. Further, the content section may be automatically populated with information associated with the printing run such as print timing information, publisher information, printer information, etc.
• Content detail information210 may be added and/or modified at any time following the generation of the electronic document. Added detail information can include adding hyperlinks to content sections, adding rich media, adding statistical information such as number of times a hyperlink is accessed, how users are using the launch point (frequency, click-throughs, purchases based on launch points, etc.), the correlation between launch points and accessed associated content, etc.
• In another embodiment, a data structure for storing print item IDs and actionable elements may comprise one or more of the following, in this or a different order: print; structure; publication; title; issue; version; signature; page; content or sub-content item; actionable element. The content may be, for example, an advertisement, editorial, product description, etc. The content may be, for example, an image, a graph, a chart, a logo, etc.
• FIG. 10 is a flowchart of a method for processing actionable elements related to a multi-page printed piece, according to one illustrative embodiment. At ablock1000, an input device is configured to receive input data which identifies the multi-page printed piece. The input device may be a camera, text box on a screen, RFID tag, etc. At ablock1002, a transceiver circuit is configured to transmit a request for data based on the received input data. At ablock1004, the transceiver is configured to receive response data, which may take a variety of forms, formats, or protocols. The response data includes data that can be used to provide actionable elements on a screen of the mobile computing device. At ablock1006, a processing circuit coupled to the transceiver circuit is configured to generate, based on the response data, actionable elements for each of a plurality of different pages in the multi-page printed piece for display. The actionable elements are configured to be selectable by a user to retrieve and/or display additional content related to the selected actionable element.
• FIG. 11 is a flowchart of a method for processing actionable elements related to a printed item or other physical object, according to an illustrative embodiment. At ablock1100, an input device is configured to receive input data which identifies the printed item. At ablock1102, a transceiver circuit coupled to the processing circuit is configured to transmit a request for data and (at a block1104) to receive response data. At ablock1106, a processing circuit coupled to the transceiver circuit is configured to generate, based on the response data, an actionable element previously associated with the printed item for display. The actionable element represents the printed text and is displayed in a format different than a format of the printed text.
• The actionable element may relate to the printed text, include portions of the printed text, or otherwise be associated with the printed text. The actionable element, while representing the printed text, may be displayed in a different format, for example, in a different font, with hypertext underlining, separated from a corresponding or underplayed image, or otherwise differently formatted. As another example, the actionable element may be displayed without displaying a full image of the printed item having the printed text. In one embodiment, but not downloading a full image of the printed item (at full resolution, size, and/or scale), communication bandwidth may be reduced.
• FIG. 12 is a flowchart of a method for processing a request for actionable elements, according to an illustrative embodiment. At ablock1200, a memory is configured to identifying information (e.g., IDs, image characteristics, titles, etc.) for a plurality of different multi-page printed pieces. The different multi-page printed pieces may be, for example, a magazine from a first publisher and a magazine from a second, different publisher. At ablock1202, a memory is configured to store associated actionable elements for each printed piece. The actionable elements may be indexed or otherwise associated with the printed pieces in memory. The memory may store the actionable elements indexed by pages in the multi-page printed piece which are pre-associated with the actionable elements. Pre-association may occur at any time before the request for actionable elements is received. Alternatively, association of actionable elements and pages may occur dynamically or “on the fly” or after receiving the request for actionable elements. For example, thesystem150 may be configured to select data elements, components, metadata, data files, or other portions of a complete set of data for a printed page or object. These data elements may comprise substantially less memory and bandwidth than the complete set of data, thereby making them more readily downloadable. These data elements may be customizable based on search query data, thereby making them more relevant or pertinent to the particular end user.
• At a block1204, a processing circuit on a server side is configured to receive via a network interface circuit a request for actionable elements for the multi-page printed piece, the request having an ID. The request may come from a mobile computing device, which acquires the ID using a discovery mechanism. At ablock1206, the processing circuit is configured to compare the ID in the request to the IDs stored in the memory. At ablock1208, the processing circuit is configured to retrieve the actionable elements for different pages of the multi-page printed piece from the memory based on the comparison. At ablock1210, the processing circuit is configured to send the retrieved actionable elements via the network interface circuit, back to the requesting device.
• FIG. 13 is a flowchart of a method for building an infrastructure of actionable elements related to different printed items or physical objects, according to one illustrative embodiment. At a block1300, a memory is configured to store IDs for a plurality of different printed items to be printed for different customers. At a block1302, a processing circuit is configured to receive, for each printed item, a selection of a discovery mechanism for discovering the printed item from a plurality of different discovery mechanisms. The discovery mechanism selection may include any of receiving a selection among different discovery technologies, receiving an upload of a bar code, receiving an upload of image recognition data or a selection of an image or portion thereof to be recognized, receiving a selection of a code for use with an RFID tag, or other selection actions. At ablock1304, the processing circuit is configured to receive, for each printed item at least one actionable element, the actionable element providing access to content or functionality supplemental to content printed on the printed item. At ablock1306, the processing circuit is configured to store the selection of the discovery mechanism and the actionable element in the memory in association with the ID for each of the plurality of different printed items to be printed for different customers (e.g., marketers, publishers, etc.).
• FIG. 14 is a flowchart of a method for processing a request for supplemental content, according to an illustrative embodiment. At ablock1400, the method comprises receiving a request for content supplemental to printed content in a multi-page printed piece. The request may comprise a code retrieved from the multi-page printed piece using a discovery mechanism. At ablock1402, the method comprises retrieving from a memory device the supplemental content, wherein the supplemental content comprises different supplemental content items for different pages of the multi-page printed piece. For example, the supplemental content items may comprise at least one of a hyperlink, a phone call item, an augmented reality item and a video item. At ablock1404, the method comprises sending the retrieved supplemental content for use with a mobile computing device.
• In an alternative embodiment, the processing circuit is further configured to send an application to the mobile computing device to be used by the mobile computing device to retrieve the code from the multi-page printed piece. The application may be software embodied on a tangible medium, downloadable to the mobile computing device (for example, through an “app store”), or another type of computer application. The application may be configured to perform any of the functions described herein, such as retrieving code from the multi-page printed piece in a discovery process.
• FIG. 15 is a flowchart illustrating a method for generating digital elements associated with a printed piece or physical object based on received variant information, according to one illustrative embodiment. In one exemplary embodiment, a printed publication identification request, entered using a home screen for example, may be made by an end-user through the use of an end-user device12 in ablock1502. Regardless of the form of the request or how it is made, the request can be provided by the end-user to a processing circuit, either directly or indirectly. The request can also be provided through a network, such as a phone or data network that can be wired or wireless. Likewise, the request may be received by a server-side processing circuit, either directly or indirectly from the end-user or end-user device, via the network.
• Upon receiving the request, the request may be compared to the stored content indatabase118 and the information related thereto. As an example, a search of a publication by title will be compared to title information of the stored content. Alternatively, a request coming from a bar code may include publication information (title, version, etc.) that is compared to the same type of information of the stored content. The comparison will result in a list of potential matches, which can be sent to and received by the end-user via the end-user device for confirmation or selection. This may result from a request that does not include enough information to specifically identify a specific printed publication or version thereof. Upon such confirmation or selection, the printedpublication100 and an associatedelectronic document200 is identified in astep1504.
• In astep1506, the processing circuit is configured to determine whether the electronic document or digital elements associated with the printed piece include any content variants. If so, in astep1506, the processing circuit is configured to determine the nature of the variants and to determine available variant resolution information. For example, where the electronic document is a clothing catalog that includes a regional variant for northern states and a regional variant for southern states, the processing circuit may be configured to attempt to obtain additional information regarding the source of the request received instep1502. Steps may include accessing stored information regarding thedevice12 or a user ofdevice12, utilizing GPS functionality of thedevice12, etc. Variant resolution instep1508 may also include displaying the variants for selection by the user.
• Variant resolution instep1508 allows information fromdevice12 and/or end user opt-in identification to be used to generate customized publication augmentation information. This provides a flexible and diverse method of publication content discovery for the end user to enhance the print viewing experience.
• Advantageously, information available throughdevice12, a user profile associated withdevice12, and information associated with the printed publication itself can be used to facilitate variant resolution. Information available throughdevice12 can include a device location, a device type, a device model, device usage statistics, etc. The user profile may be provided by the user and/or may be culled from existing information sources. Publication information for the printed publication may include identifying the variant of the printed publication, identifier of a subscriber or purchaser of the printed publication, etc. The variant of the electronic document may be different from a variant of the printed publication depending on the inputs used instep1508. The user profile may include user-set preferences. For example, if the user is using a device that is not well suited for a telephone call, the user can request that any phone call actionable elements not be presented on its screen.
• Advantageously, generating an electronic document during the printing of a printed publication may allow the system to utilize the same or a similar set of data structures and workflows from the printed publication to deliver/distribute a multi-channel experience (print and digital) through the electronic document. This efficiency may reduce problems of double submissions and workflows which cause delay, higher incidence of error, reduced labor requirements, etc.
• Following resolution of the variants instep1508, a customized representation of the electronic document may be presented to the user ofdevice12 instep1510. Generating a customized electronic document may include stitching together the signatures including digital content to generate a digital version of the printed publication at the time of therequest1508 based on variant information that is available also at the time of the request.
• Presenting the customized electronic document instep1510 can further include displaying content associated with the generated version of printed publication in astep1512. The displayed content linked to printed media may represent anything that can be displayed on the mobile device: URLs, phone numbers, email addresses, video, audio, etc.
• The method offlowchart1500 can be used to display the printed media and linked content from a single end-user request submission. For example, the single end-user request submission could be generated by manually searching, using the device to conduct imaging or image recognition within that publication, scanning a barcode such as a UPC code or QR code, interaction with a tag such as a Near Field Communication (NFC) tag, or other means. The launch point used to generate the request no longer needs to correlate to individual pages or content links (though it may in alternative embodiments), but instead may correlate to the entire printed media piece or groups of related pages and content within the printed media piece. The launch point now may provide a proxy for use in print media marketing based on single images or ads. For example, the launch point may be used as a catalog proxy for the entire catalog, a launch point on an item as a proxy to a complete catalog and so on.
• Advantageously, the method offlowchart1500 provides a system that has the ability to pull separate image/page files in real time to build (or stitch) a version of a publication at the time of a request as opposed to grabbing a single file of the complete publication. This process gives an advantage of reducing prep and handling time and also reduces the amount of data to be stored (i.e., no need to store multiple complete variants), especially as it relates to publications with numerous versions.
• FIG. 16 is a flowchart illustrating a method for generating digital elements associated with a printed piece or physical object based upon variant information, according to an illustrative embodiment. The method offlowchart1600 includes the process of pulling together separate images, or pages, or links for mobile content fromdatabase118 in real-time at the time the user accesses the publication. This content can be based on personalized data, like either the demographic or geographic attributes of the user accessing the work as described above.
• In astep1602,system150 receives input parameters related to the printed publication and the end user. Input Parameters may be any one or combination of the known print parameters—title, publication or in-home date, page and advertising version information, page sequence, mail address or newsstand location, personalization, etc.—and mobile devices parameters—smartphone or tablet type, geo-location, user opt-in selection, method of discovery applied, etc. Geographical location for print media may be determined by the final physical destination, be it the postal address or newsstand regional location. The print postal zip code, carrier route, and newsstand location information may be used to correlate to mobile device end user interaction, geo-location, and opt-in profile. Device characteristics may include, for example, whether a device supports web browsing, multi-media support such as video and audio, and whether a device supports autonomous applications. The device, along with the stitched together and downloaded information and Internet links, can provide an augmented content experience that serves as a companion to print media. User characteristics may include characters of those individuals and publishers who wish to have or provide an enhanced print media experience. Users reading a print media piece may use their mobile device to interact with content and information called-out and/or linked within the print. Once a user has accessed a publication version, they can scroll to or activate any page or link within the publication.
• Based on the received input parameters, thesystem150 can generate a listing of content detail information in astep1604. The content detail information can include associated content from the electronic document or supplemental content defined based on the structure of the printed publication. The content from the electronic document may be defined based on the structure of the printed publication because of the method used to generate the electronic document, described above. Following structure definition, content may be associated with the content of the electronic document without modifying the underlying structure.
• In astep1606, the processing circuit on the server or device side can be used to identify and select content variants based on the device information, printed publication information, and user information received instep1602. Selection of content variants may include selection of associated content rather than selection of the content in the printed publication.
• In astep1608,device12 can be configured to access an associated content listing, similar to the listing shown and described above with reference toFIGS. 5 and 6. The listing of associated content can include a photograph of the content from the printed publication along with additional functionality such as a web link, a hyperlink, mapping functionality, etc.
• Access to the content may be in the form of a digital reproduction of the specific version of the printed publication, a customized version of the publication based upon the segmented information known about the end user, or otherwise providing linking or interactivity with the content from the printed publication. This may be through a URL, web page, or other digital destination or content, including GPS, content information, invitation for electronic or digital communications, etc. The end users also may desirably distribute or share the content from the printed publication with other users. Access to the content may be obtained through the display, through an audio output circuit, through haptic feedback, etc.
• FIG. 17 is a computer implemented augmentation system for presenting digital information related to the contents of a printed piece or object based on received information identifying the printed piece or object, according to an illustrative embodiment. The description of the components below may apply to other components shown or described in other embodiments presented herein.
• Augmentation system1710 includes aninformation processor1712, one or moreidentification input devices1714, auser interface1716, and amemory1718. Althoughsystem1710 is shown as including a single system including specific components associated with specific function, it should be understood that the functionality described herein may be performed using multiple systems and more, fewer and/or a different arrangement of components, and that functionality described as being performed by any specific component may alternatively be performed by any other component.
• The functional components ofsystem1710 are implemented using hardware including a processing circuit implementing theinformation processor1712, asystem memory1718 for storing instructions for implementing theinformation processor1712, and a system bus that couples various system components, including the system memory, to the processing circuit. Further, the described components may be implemented using hardware and/or software embodied on a tangible medium.
• The processing circuit may comprise or be a portion of any type of computer processor comprising suitable analog and/or digital circuit components, such as a microprocessor, microcontroller, application-specific integrated circuit, programmable logic, etc., configured to implement the functionality described herein. The processor also may be local or network based.
• The system memory may include computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) and/or random access memory (RAM).System1710 may also include other removable/non-removable, volatile/nonvolatile computer storage media, such as a local or cloud-based drive including theinformation database1718.
• Information processor1712 may comprise software embodied on a computer-readable medium (such as firmware, memory, a CD-ROM, etc.) and implemented by a processing circuit configured to determine an identification of a printed piece or object and to provide information to theuser interface1716 to augment the information in the printed piece or object. Specifically, in one embodiment theprocessor1712 may be configured to receive input from aninput device1714 to allowprocessor1712 to identify a printedpiece10 and optionally a page in the printed piece. Based on the identified page and/or printed piece,processor1712 may be configured to retrieve augmentation information from thememory1718 for theuser interface1716. The augmentation information can include digital elements associated with the printed piece or object, and may include an electronic copy of the printed piece or descriptions of the content within the printed piece, text, graphics, etc., that are customized to thedevice1710, user ofdevice1710 and/or augmented content accessed in context based on the identified page.
• Input device1714 can include any type of input device or method for receiving information or an identification of a printed piece or object. Examples can include, but are not limited to a camera, a transceiver, a microphone, a keyboard including either a physical keyboard and/or a virtual keyboard implemented using thedisplay1716, haptic or capacitive screens, radio frequency readers, or other electronic devices and sensors that are configured to interact with a printed piece or object. Theinput device1714 may be embodied in a wearable housing, such as eyeglasses, a wristband, a belt clip, a hat, etc.Input devices1714 can be configured to operate in combination with software implemented byprocessor1712 to perform a number of different recognition or discovery methods. For example, acamera input device1714 may capture an image of text, graphics and or code within the printed piece or object, and convey that to an image matching circuit implemented byprocessor1712. Among other possibilities, a user can use theinput device1714 to recognize a printed piece cover, content within a printed piece, a universal product code or other type of barcode displayed in the printed piece, an ISBN number, radio frequency tag or near field communication tag, text or graphics, etc. to initiate a process for identifying the printed piece. Alternatively, a user can manually enter the name of a printed piece or object using search terms to find the desired printed piece or object.
• User interface1716 can be any type of display configured to display information provided byprocessor1712, such as a liquid crystal display (LCD), an active matrix LCD, an organic light-emitting diode display, or other display types, audio device or speaker, haptic screen, or other interface that can convey information to any one or all of the five senses.
• Augmentation system1710 may be configured to access or display digital elements associated with the piece generated during the printing of the printed publication. The digital elements associated with the document may be augmented or supplemented with additional functionality related to the content contained in the printed publication. The augmentation may be provided by functional blocks overlaid on the display of the electronic document, by displaying a table of contents including available functional content where the table of contents is structured to mirror the content contained in the printed publication, by providing supplemental content may not display information such as an audio file playback, haptic feedback, etc., or any combinations thereof, etc.
• According to one embodiment, a computer implemented method for providing digital elements associated with a printed piece or other object comprises: receiving a request from a requester for a digital element associated with the printed piece or object, the request including information to identify the printed piece or object; selecting one or more content variants associated with the printed piece, object or requester of the digital element; generating the digital element to include or be based on the one or more content variants; and sending the digital elements to the requester.
• The request may further include content variants such as location data indicating a geographic origin of the request, information about the device or user of the device. Generating a digital element may include assembling the digital elements based on such content variants. Each content variant may be selected based on at least one of the printed piece identification, object information, user information, device information, and request information.
• In one embodiment, generating a digital element associated with the printed piece or object may include generating a table of contents based on the selected content variants. The digital elements or representation associated with the printed piece or object may be unique from the printed piece or object itself based on the selected content variants. Ultimately, this allows a user to experience a version of the printed publication or object relevant to that user, and allows brand-owners and content creators to deliver more useful, targeted and personalized messages and offers and calls for action to users.
• According to another embodiment, a computer implemented method for providing a representation of—or digital elements associated with—a printed piece or object based on received input data may comprise: receiving identification information related to a printed piece or object; matching the received identification information to a printed piece or object; identifying content associated with the printed piece or object, the content optionally being structured based on the structure of the printed piece or object description; and sending the associated content for use on a user interface. The associated content may be received by and/or downloaded to a computing device; and the associated content may be communicated via the user interface.
• The printed piece or object may include one or a plurality of launch points, each launch point being associated with unique associated content. The digital elements associated with these launch points may be configured to include one or more hyperlinks based on the information associated with these launch points. The launch points may comprise a graphic and/or text, such as those found on the cover or page of certain printed pieces. Other launch points may include a radio frequency or near field communication (NFC) tag, a quick response (QR) code, a universal product code (UPC), a barcode, an international standard book number (ISBN), a wireless transmitter, a product shape or distinctive element, etc. The radio frequency, near field communication (NEC), or wireless devices may be active or passive devices. The information obtained through the input device from the launch point is processed to information that can be sent for comparison with information about the launch point, for example through a database look-up table. Once a positive identification of the printed piece or object is made, the digital elements associated with the printed piece or object may be sent to the user interface and/or downloaded.
• According to another embodiment, a computer implemented system for providing a customized digital information related to a printed piece or object using a computing device may comprise: one or more input devices for receiving information for identifying a printed piece or object and at least one of a device characteristic, a user characteristic and a printed piece characteristic or object characteristic. An electronic document or a digital element generator implemented by a processor on a computing device may be configured to select one or more content variants associated with the printed piece or object. The content variants may be selected based on one of a device characteristic, a user characteristic, and a printed publication characteristic. The processor may further be configured to generate a digital representation or digital elements of the printed publication to include the plurality of content variants. An electronic display of the computing device may be configured to display the digital representation of the printed piece or object. The device characteristic may include GPS or other location data indicating a request geographic origin. Generating digital information related to the printed piece or object may include assembling a plurality of content variants selected based on the GPS data. Generating digital information related to the printed piece or object may further include generating arable of contents based on one or more of the content variants. The digital information related to the printed piece or object may be unique from the printed publication based on the selected content variants.
• While some embodiments are described with reference to printed pieces or printed publications, other embodiments may be used with non-printed objects, such as perhaps a purse or handbag, shaped items such as bottles or containers, items having a distinctive or recognizable feature, or objects that may otherwise be recognizable using a discovery mechanism. The embodiments herein may allow these brand owners or advertisers to deliver additional content such as actionable elements relating to everyday objects to the user.
• The invention has been described in terms of several preferred embodiments. It will be appreciated that the invention may otherwise be embodied without departing from the fair scope of the invention.

Claims (20)

What is claimed is:

1. A computing device for displaying actionable elements related to a multi-page printed piece, comprising:

an input device configured to receive input data which identifies the multi-page printed piece;

a transceiver circuit configured to transmit a request for data based on the received input data and to receive response data;

a processing circuit coupled to the transceiver circuit and configured to generate, based on the response data, actionable elements for each of a plurality of different pages in the multi-page printed piece for display, the actionable elements being selectable by a user to interact with the additional content related to the selected actionable element.

2. The computing device ofclaim 1, wherein the actionable elements comprise at least one of hyperlinks to server-based content or a phone call element configured to make a phone call using the computing device.

3. The computing device ofclaim 1, further comprising a display configured to display actionable elements for the plurality of different pages simultaneously on a single screen on the display.

4. The computing device ofclaim 1, wherein the processing circuit is configured to generate display data representing a plurality of different multi-page printed pieces for simultaneous display on a single screen.

5. The computing device ofclaim 4, wherein in response to selection of one of the multi-page printed pieces being simultaneously displayed, a plurality of actionable elements for the selected piece are displayed.

6. The computing device ofclaim 1, wherein the actionable elements are arranged for display based on page numbers of the multi-page printed piece relating to each of the actionable elements.

7. The computing device ofclaim 1, wherein the actionable elements for the plurality of different pages are generated based on a single input data identifying the multi-page printed piece.

8. The computing device ofclaim 1, wherein the input device comprises a camera and the input data is received from an image, or wherein the input device comprises an RFID circuit.

9. The computing device ofclaim 1, wherein the input device comprises a displayed text input box configured to receive textual input from a user.

10. The computing device ofclaim 1, wherein the actionable elements correspond to printed text in the multi-page printed piece, wherein the actionable elements are displayed in a format different than a format of the printed text.

11. A computing device for displaying actionable elements related to a printed item, the printed item having printed information, comprising:

an input device configured to receive input data which identifies the printed item;

a transceiver circuit coupled to the processing circuit and configured to transmit a request for data and to receive response data;

a processing circuit coupled to the transceiver circuit and configured to generate, based on the response data, an actionable element previously associated with the printed item for display, wherein the actionable element represents the printed information and is displayed in a format different than a format of the printed information.

12. The computing device ofclaim 11, wherein the actionable elements comprise at least one of a hyperlink to server-based content or a phone call element configured to make a phone call using the computing device.

13. The computing device ofclaim 11, wherein the actionable element is displayed without displaying a full image of the printed item having the printed information.

14. The computing device ofclaim 11, wherein a full image of the printed item is not downloaded to the computing device in response to the request data.

15. The computing device ofclaim 11, wherein the actionable element represents printed information on an outdoor advertising sign.

16. A computerized method of displaying on a computing device actionable elements related to a multi-page printed piece, comprising:

receiving at an input device input data which identifies the multi-page printed piece;

sending a request for data based on the input data;

receiving response data based on the request for data;

generating, at a processing circuit, based on the response data, actionable elements for each of a plurality of different pages in the multi-page printed piece for display, wherein the actionable elements are selectable by a user to retrieve and/or display additional content related to the selected actionable element.

17. The computerized method ofclaim 16, further comprising retrieving server-based content in response to user selection of one of the actionable elements.

18. The computerized method ofclaim 16, further comprising sending display data to display actionable elements for the plurality of different pages simultaneously on a single screen on the display.

19. The computerized method ofclaim 16, wherein the actionable elements are arranged for display based on page numbers of the multi-page printed piece relating to each of the actionable elements.

20. The computerized method ofclaim 16, further comprising:

receiving an image of the printed piece using a camera of the computing device; and

providing the input data based on an image recognition process performed on the image of the printed piece.

Images