US20190019521A1 - System and method for utilizing audio encoding for measuring media exposure with environmental masking - Google Patents

Abstract

An audio beacon system, apparatus and method for collecting information on a panelist's exposure to media. An audio beacon is configured as on-device encoding technology that is operative in a processing device (e.g., cell phone, PDA, PC) to enable the device to encode an environmental sound and transmit it for a predetermined period of time. The acoustically transmitted data is received and processed by a portable audience measurement device, such as Arbitron's Personal People Meter™ (“PPM”), or other specially equipped portable device to enable audience measurement systems to achieve higher levels of detail on panel member activity and greater association of measurement devices to their respective users.

Description

RELATED APPLICATIONS
• The present application is a continuation-in-part of U.S. patent application Ser. No. 12/425,464, titled “System and Method for Utilizing Audio Beaconing in Audience Measurement” filed Apr. 17, 2009, and U.S. patent application Ser. No. 12/425,556, titled “System and Method for Utilizing Supplemental Audio Beaconing in Audience Measurement,” also filed on Apr. 17, 2009. Both applications are assigned to the assignee of the present application and are incorporated by reference in their entireties herein
TECHNICAL FIELD
• The present disclosure relates to systems and processes for communicating and processing data, and, more specifically, to communicate media data exposure that may include coding that provides media and/or market research.
BACKGROUND INFORMATION
• The use of global distribution systems such as the Internet for distribution of digital assets such as music, film, computer programs, pictures, games and other content continues to grow. In many instances, media offered via traditional broadcast mediums is supplemented through similar media offerings through computer networks and the Internet. It is estimated that Internet-related media offerings will rival and even surpass traditional broadcast offerings in the coming years.
• Techniques such as “watermarking” have been known in the art for incorporating information signals into media signals or executable code. Typical watermarks may include encoded indications of authorship, content, lineage, existence of copyright, or the like. Alternatively, other information may be incorporated into audio signals, either concerning the signal itself, or unrelated to it. The information may be incorporated in an audio signal for various purposes, such as identification or as an address or command, whether or not related to the signal itself.
• There is considerable interest in encoding audio signals with information to produce encoded audio signals having substantially the same perceptible characteristics as the original unencoded audio signals. Recent successful techniques exploit the psychoacoustic masking effect of the human auditory system whereby certain sounds are humanly imperceptible when received along with other sounds.
• Arbitron has developed a new and innovative technology called Critical Band Encoding Technology (CBET) that encompasses all forms of audio and video broadcasts in the measurement of audience participation. This technology dramatically increases the both the accuracy of the measurement and the quantity of useable and effective data across all types of signal broadcasts. CBET is an encoding technique that Arbitron developed and that embeds identifying information (ID code) or other information within the audio portion of a broadcast. An audio signal is broadcast within the actual audio signal of the program, in a manner that makes the ID code inaudible, to all locations the program is broadcast, for example, a car radio, home stereo, computer network, television. etc. This embedded audio signal or ID code is then picked up by small (pager-size) specially designed receiving stations called Portable People Meters (PPM), which capture the encoded identifying signal, and store the information along with a time stamp in memory for retrieval at a later time. A microphone contained within the PPM receives the audio signal, which contains within it the ID code.
• Further disclosures related to CBET encoding may be found in U.S. Pat. No. 5,450,490 and U.S. Pat. No. 5,764,763 (Jensen et at.) in which information is represented by a multiple-frequency code signal which is incorporated into an audio signal based upon the masking ability of the audio signal. Additional examples include U.S. Pat. No. 6,871,180 (Neuhauser et al.) and U.S. Pat. No. 6,845,360 (Jensen et al.), where numerous messages represented by multiple frequency code signals are incorporated to produce and encoded audio signal. Other examples include U.S. Pat. No. 7,239,981 (Kolessar et al.). Each of the above-mentioned patents is incorporated by reference in its entirety herein.
• The encoded audio signal described above is suitable for broadcast transmission and reception and may be adapted for Internet transmission, reception, recording and reproduction. When received, the audio signal is processed to detect the presence of the multiple-frequency code signal. Sometimes, only a portion of the multiple-frequency code signal, e.g., a number of single frequency code components, inserted into the original audio signal, is detected in the received audio signal. However, if a sufficient quantity of code components is detected, the information signal itself may be recovered.
• Other means of watermarking have been used in various forms to track multimedia over computer networks and to detect if a user is authorized to access and play the multimedia. For certain digital media, metadata is transmitted along with media signals. This metadata can be used to catty one or more identifiers that are mapped to metadata or actions. The metadata can be encoded at the time of broadcast or prior to broadcasting. Decoding of the identifier may be performed at a digital receiver. Other means of watermarking include the combination of digital watermarking with various encryption techniques known in the art.
• While various encoding and watermarking techniques have been used to track and protect digital data, there have been insufficient advances in the fields of cross-platform digital media monitoring. Specifically, in cases where a person's exposure to Internet digital media is monitored in addition to exposure to other forms of digital media (e.g., radio, television, etc.), conventional watermarking systems have shown themselves unable to effectively monitor and track media exposure. Furthermore, there is a need to integrate exposure to digital media across platforms where the digital media includes formats that are not traditionally subject to audio encoding. Moreover, there is a need in the art to properly “mask” such signals using environmental sounds and/or sounds native to a device that is conducting beaconing processes.
SUMMARY
• Accordingly, an audio beacon system, apparatus and method is disclosed for collecting information on a panelist's exposure to media. Under a preferred embodiment, the audio beacon is configured as on-device encoding technology that is operative in a panelist's processing device (e.g., cell phone, PDA, PC) to enable the device to acoustically transmit user/panelist data for a predetermined period of time. The acoustically transmitted data is received and processed by a portable audience measurement device, such as Arbitron's Personal People Meter™ (“PPM”) or specially equipped cell phone, laptop etc., to enable audience measurement systems to achieve higher levels of detail on panel member activity and greater association of measurement devices to their respective panelists. Additionally, the acoustic transmissions are configured to utilize environmental sounds that are advantageous in being less obtrusive to users.
• Additional features and advantages of the various aspects of the present disclosure will become apparent from the following description of the preferred embodiments, which description should be taken in conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1A is a block diagram illustrating a portion of an audio beaconing system under one exemplary embodiment;
• FIG. 1B is a block diagram illustrating another portion of an audio beaconing system under the embodiment illustrated inFIG. 1A;
• FIG. 2 is a tabular illustration of an audio beaconing and audio matching process under another exemplary embodiment;
• FIG. 3 illustrates a block diagram of a server-side encoding process under yet another exemplary embodiment;
• FIG. 4 illustrates an exemplary watermarking process for a digital media file suitable for use in the embodiment ofFIGS. 1A-B;
• FIG. 5 illustrates a block diagram of a client-side encoding process under yet another exemplary embodiment;
• FIG. 6 illustrates an exemplary audio waveform having encoded data therein; and
• FIG. 7 illustrates an exemplary device configured to select environmental sounds for audio transmission.
DETAILED DESCRIPTION
• FIG. 1A is an exemplary block diagram illustrating a portion of anaudio beaconing system150 under one embodiment, where aweb page110 is provided by a page developer and published oncontent server100. The web page preferably contains an embeddedvideo player111 and audio player112 (that is preferably not visible), together with an application programming interface (API)113. Other content114 (e.g., HTML, text, etc.) is also provided onweb page110, which may or may not be coupled throughAPI113.API113 is preferably embodied as a set of routines, data structures, object classes and/or protocols provided by libraries and/or operating system services in order to support thevideo player111 andaudio player112. Additionally, theAPI113 may be language-dependent (i.e. available only in a particular programming language) or language-independent (i.e., can be called from several programming languages, preferably an assembly/C-level interface). Examples of suitable API's include Windows API, Java Platform API, OpenGL, DirectX, Simple DirectMedia Layer (SDL), YouTube API, Facebook API and iPhone API, among others.
• In one preferred embodiment,API113 is configured as a beaconing API object. Depending on the features desired, the API object may reside on an Audience Measurement (AM)server120, so that the object may be remotely initialized, thus minimizing the objects software's exposure to possible tampering and to maintain security. Alternately, the API object can reside on thecontent server100, where the API object may be initialized under increased performance conditions.
• When initialized,API113 can communicate the following properties: (1) the URL of the page playing the media, (2) URL of the media being served on the page, (3) any statically available media metadata, and (3) a timestamp. It is understood that additional properties may be communicated inAPI113 as well. In one configuration ofFIG. 1 A an initialization request is received byAPI113, to create a code tone that is preferably unique for each website and encode it on a small inaudible audio stream. Alternatively, theAM server120 could generate apre-encoded audio clip101, with a code tone, for each site and forward it on thecontent server100 in advance.
• The encoded audio stream would then travel fromcontent server100 to theweb page110 holdingaudio player110. In a preferred embodiment,audio player110 may be set by the page developer as an object instance, where the visible property ofplayer110 is oriented as “false” or set to a one-by-one dimension in order to minimize the visual interference of the audio player with the web page. The encoded audio stream may then be played out in parallel with the media content being received from theweb page110. The encoded audio stream would preferably repeat at predetermined time periods through an on-device beacon131 resident on auser device130 as long as the user is on the same website. Thebeacon131, would enabledevice130 to acoustically transmit the encoded audio stream so that a suitably configured portable device140 (e.g., PPM) can receive and process the encoded information.Beacon131 could be embedded into an audio player resident on a web page being viewed inside the browser onuser device130, or may be a stand-alone application onuser device130.
• A simplified example further illustrates the operation of thesystem150 ofFIGS. 1A-B under an alternate embodiment.User device130 requests content (e.g., http://www.hulu.com/) fromserver100. When the content is received inuser device130,PC meter software132 collects and transmits web measurement data toInternet measurement database141. One example of a PC meter is comScore's Media Metrix™ software; further exemplary processes of web metering may be found in U.S. Pat. No. 7,493,655, titled “Systems for and methods of placing user identification in the header of data packets usable in user demographic reporting and collecting usage data” and U.S. Pat. No. 7,260,837, titled “Systems and methods for user identification, user demographic reporting and collecting usage data usage biometrics”, both of which are incorporated by reference in their entirety herein.
• As web measurement data is collected byPC meter132,beacon131 acoustically transmits encoded audio, which is received byportable device140. In the exemplary embodiment, the encoding for the beacon transmission may include data such as a timestamp, portable device ID, user device ID, household ID, or any similar information. In addition to the beacon data,portable device140 additionally receives multimedia data such as television andradio transmissions142, which may or may not be encoded, at different times. If encoded (e.g., CBET encoding), portable device can forwardtransmissions142 to audio matching server160 (FIG. 1B) for decoding and matching withaudio matching database161. Iftransmissions142 are not encoded,portable device140 may employ sampling techniques for creating audio patterns or signatures, which may also be transmitted toaudio matching server160 for pattern matching using techniques known in the art.
• Audio beacon server150, shown inFIG. 1B, receives and processes/decodes beacon data fromportable device140. Under an alternate embodiment, it is possible to combineaudio matching server160 andaudio beacon server150 to collectively process both types of data. Data fromAudio beacon server150 andaudio matching server160 is transmitted toInternet measurement database141, where the web measurement data could be combined with audio beacon data and data from the audio matching server to provide a comprehensive collection of panelist media exposure data.
• Under another exemplary embodiment, the video and audio players ofwebpage110 are configured to operate as Flash Video, which is a file format used to deliver video over the Internet using Adobe™ Flash Player. The Flash Player typically executes Shockwave Flash “SWF” files and has support for a scripting language called ActionScript, which can be used to display Flash Video from an SWF file. Because the Flash Player runs as a browser plug-in, it is possible to embed Flash Video in web pages and view the video within a web browser. Commonly, Flash Video files contain video bit streams which are a variant of the H.263 video standard, and include support for H.264 video standard (i.e., “MPEG-4 part 10”, or “AVC”). Audio in Flash Video files (“FLV”) is usually encoded as MP3, but can also accommodate uncompressed audio or ADPCM format audio.
• Continuing with the embodiment, video beacons can be embedded within an action script that will be running within the video Flash Player's run time environment onweb page110. When an action script associated withweb page110 gets loaded as a result of the access to the page, the script gets activated and triggers a “video beacon”, which extracts and store URL information on a server (e.g., content server100), and launches the video Flash Player. By inserting an audio beacon in the same action script, the audio beacon will be triggered by the video player. Once triggered, the audio beacon may accessAM server120 to load a pre-recorded audio file containing a special embedded compatible code (e.g., CBET). This pre-recoiled audio file would be utilized forbeacon131 to transmit for a given period of time (e.g., every x seconds).
• As a result, thebeacon131 audio player runs as a “shadow player” in parallel to the video Flash Player. If aportable device140 is in proximity touser device130,portable device140 will detect the code and reports it toaudio beacon server150. Depending on the level of cooperation between the audio and video beacon, the URL information can also be deposited ontobeacon server150 along with codes that would allow an audience measurement entity to correlate and/or calibrate various measurements with demographic data.
• Under the present disclosure, media data may be processed in a myriad of ways for conducting customized panel research. As an example, eachuser device130 may install on-device measurement software (PC meter132) which includes one or more web activity monitoring applications, as well asbeacon software131. It is understood that the web activity monitoring application and the beacon software may be individual applications, or may be merged into a single application.
• The web activity monitoring application collects web activities data from the user device130 (e.g., site ID, video page URL, video file URL, start and end timestamp and any additional metadata about videosite information, URL information, time, etc.) and additionally assigns a unique ID, such as a globally unique identifier or “GUID”, to each device. For thebeacon131, a unique composite ID may be assigned including a household ID (“HHID”) and a unique user device ID for each device in the household up to 10 devices for a family), as well as a portable device ID (PPMID). Panelist demographic data may be included for each web activity on the device.
• Continuing with the example,beacon131 emits an audio beacon code (ABC) for device in the household by encoding an assigned device ID number and acoustically sending it toportable device140 to identify the device. Further details on the encoding is provided below.Portable device140 collects the device ID and sends it to a database along with HHID and/or PPM ID and the timestamp. Preferably, a PPMID is always mapped to a in the backend; alternately an HHID can be set within each PPMID.
• The web activity monitoring and beacon applications may pass information to each other as needed. Both can upload information to a designated server for additional processing. A directory of panelists' devices is built to contain the GUID, HHID, and device ID for panel, and the directory could be used to correlate panelist demographic data and web measurement data.
• Turning toFIG. 2, a tabular illustration of an audio beaconing and audio matching process under another an exemplary embodiment is provided. Specifically, the table illustrates a combination of audio beaconing and audio matching and its application to track a video on a content site, such as Hulu.com.FIG. 2.Timeline200 shows in sections a scenario where a user/panelist plays a ten minute video on Hulu.com.Activities201 shows actions taken inuser system150 where a video is loaded in theuser device130, and played. At the 5 minute mark (301 sec.) a 15 second advertisement is served. At the conclusion of the advertisement (316 sec.), the video continues to play until its conclusion (600 sec.).
• During this time,audio beacon activities202 are illustrated, where, under one embodiment, on-device beacon131 transmits continuous audio representing the website (Hulu.com). In addition, beacon also transmits a timestamp, portable device ID, user device ID, household ID and/or any other data in accordance with the techniques described above. Under an alternate embodiment shown in203, additional data may be transmitted in the beacon to include URLs and video ID's when a video is loaded and played. As the advertisement is served, an event beacon, which may include advertisement URL data, is transmitted. At the conclusion of the video, a video end beacon is transmitted to indicate the user/panelist is no longer viewing specific media.
• When the video and advertisement is loaded and played, additional audio matching may occur in theportable device140, in addition with audio matching processes explained above in relation toFIGS. 1A-B. Referring toaudio matching events204,portable device data205 and end-user experience206 ofFIG. 2, portable device data (e.g., demographic ID data) is overlayed along with site information (URL, video ID, etc.) when a video is loaded. When the video is played, audio signatures may be sampled periodically byportable device140, until a content match is achieved. The audio signatures may be obtained through encoding, pattern matching, or any other suitable technique. When a match is found, portable device data is overlayed to indicate that a content match exists. Further signature samples are taken to ensure that the same content is being viewed. When an advertisement is served, the sampled signature will indicate that different content is being viewed, at which point the portable device data is overlayed in the system. When the video resumes, the audio signature indicates the same video is played, and portable device data is overlayed through the end of the video as shown inFIG. 2.
• As explained above, signature sampling/audio matching allows thesystem150 to identify and incorporate additional data on the users/panelists and the content being viewed. Under a typical configuration, the content provider media (e.g., Hulu, Facebook, etc.) may be sampled in advance to establish respective signatures for content and stored in a matching database (e.g., audio matching server160). Theportable device140 would be equipped with audio matching software, so that, when a panelist is in the vicinity ofuser device130, audio matching techniques are used to collect the signature, or “audio fingerprint” for the incoming stream. The signatures would then be matched against the signatures in the matching database to identify the content.
• It is understood by those skilled in the art however, that encoding techniques may also be employed to identify content data. Under such a configuration, content is encoded prior to transmission to include data relating to the content itself and the originating content site. Additionally, data relating to possible referral sites (e.g., Facebook, MySpace, etc.) may be included. Under one embodiment, a content management system may be arranged for content distributors to choose specific files for a corresponding referral site.
• For the media data encoding, several advantageous and suitable techniques for encoding audience measurement data in audio data are disclosed in U.S. Pat. No. 5,764,763 to James M. Jensen, et al., which is assigned to the assignee of the present application, and which is incorporated by reference herein. Other appropriate encoding techniques are disclosed in U.S. Pat. No. 5,579,124 to Aijala, et al., U.S. Pat. Nos. 5,574,962, 5,581,800 and 5,787,334 to Fardeau, et al., U.S. Pat. No. 5,450,490 to Jensen, et al., and U.S. patent application Ser. No. 09/318,045, in the names of Neuhauser, et al., each of which is assigned to the assignee of the present application and all of which are incorporated by reference in their entirety herein.
• Still other suitable encoding techniques are the subject of PCI Publication WO 00/04662 to Srinivasan, U.S. Pat. No. 5,319,735 to Preuss, et al., U.S. Pat. No. 6,175,627 to Petrovich, et al., U.S. Pat. No. 5,828,325 to Wolosewicz, et al., U.S. Pat. No. 6,154,484 to Lee, et al., U.S. Pat. No. 5,945,932 to Smith, et al., PCI Publication WO 99/59275 to Lu, et al., PCT Publication WO 98/26529 to Lu, et al., and PCT Publication WO 96/27264 to Lu, et al, all of which are incorporated by reference in their entirety herein.
• Variations on the encoding techniques described above are also possible. Under one embodiment, the encoder may be based on a Streaming Audio Encoding System (SAES) that operates under a set of sample rates and is integrated with media transcoding automation technology, such as Telestream's FlipFactory™ software. Also, the encoder may be embodied as a console mode application, written in a general-purpose computer programming language such as “C”. Alternately, the encoder may be implemented as a Java Native Interface (JNI) to allow code running in a virtual machine to call and be called by native applications, where the JNI would include a JNI shared library for control using Java classes. The encoder payloads would be configured using specially written Java classes. Under this embodiment, the encoder would use the information hiding abstractions of an encoder payload which defines a single message. Under a preferred embodiment, the JNI encoder would operate using a 44.1 kHz sample rate.
• Examples of symbol configurations and message structures are provided below. One exemplary symbol configuration uses four data symbols and one end symbol defined for a total of five symbols. Each symbol may comprise five tones, with one tone coming from each of five standard Barks. One exemplary illustration of Bark scale edges (in Hertz), would be {920, 1080, 1270, 1480, 1720, 2000}. The bins are preferably spaced on a 4×3.90625 grid in order to provide lighter processing demands, particularly in cases using decoders based on 512 point fast Fourier transform (FFT). an exemplary bin structure is provided below:
• • Symbol 0: {248, 292, 344, 400, 468}
  • Symbol 1: {252, 296, 348, 404, 472}
  • Symbol 2: {256, 300, 352, 408, 476}
  • Symbol 3: {260, 304, 356, 412, 480}
  • End Marker Symbol: {264, 308, 360, 416, 484}
• Regarding message structure, an exemplary message would comprise 20 symbols, each being 400 milliseconds in duration, for a total duration of 8 seconds. Under this embodiment, the first 3 symbols could be designated as match/check criteria symbols, which could be the simple sum of the data symbols or could be derived from an error correction or cyclical redundancy check algorithm. The following 16 symbols would then he designated as data symbols, leaving the last symbol as an end symbol used for a marker. Under this configuration, the total number of possible symbols would be 4¹⁶or 4,294,967,296 symbols.
• Variations in the algorithmic process for encoding are possible as well under the present disclosure. For example, a core sampling rate of 5.5125 kHz may be used instead of 8 kHz to allow down-sampling from 44.1 kHz to be efficiently performed without pre-filter (to eliminate aliasing components) followed by conversion filter to 48 kHz. Such a configuration should have no effect on code tone grid spacing since the output frequency generation is independent of the core sampling rate. Additionally, this configuration would limit the top end of the usable frequency span to about 2 kHz (as opposed to 3 kHz under conventional techniques) since frequency space should be left for filters with practical numbers of taps.
• Under one embodiment, a 16 point overlap of a 256 point large FFT is used, resulting in amplitude updates every 2.9 milliseconds for encoding instead of every 2 milliseconds for standard CBET techniques. Accordingly, fewer large FFTs are calculated under a tighter bin resolution of 21.5 Hz instead of 31.25 Hz.
• The psychoacoustic model calculations used for the encoding algorithm under the present disclosure may vary from traditional techniques as well. In one embodiment, bin spans of the clumps may be set by Bark boundaries instead of being wholly based on Critical Bandwidth criteria. By using Bark boundaries, a specific bin will not contribute to the encoding power level of multiple clumps, which provides less coupling between code amplitudes of adjacent clumps. When producing Equivalent Large FFTs, a comparison may he made of the most recent 16 point Small FFT results to a history of squared sums to simplify calculations.
• For noise power computation, the encoding algorithm under the present disclosure would preferably use 3 bin values over a clump: the minimum bin power (MIN), the maximum bin power (MAX), and the average bin power (AVG). Under this arrangement, the bin values could be modeled as follows:

• 		IF (MAX > (2 * MIN))
  		PWR = MIN
  		ELSE
  		PWR = AVG

  
  Here, PWR may be scaled by a predetermined factor to produce masking energy.
• A similar algorithm could also be used to create a 48 kHz native encoder using a core sample rate of 6 kHz and a large FFT bin resolution of 23.4375 Hz calculated every 2.67 milliseconds. Such a configuration would differ slightly in detection efficiency and inaudibility from the embodiments described above, but it is anticipated that the differences would be slight.
• With regards to decoding, an exemplary configuration would include a software decoder based on a JNI shared library, which performs calculations up through the bin signal-to-noise ratios. Such a configuration would allow an external application to define the symbols and perform pattern matching. Such steps would be handled in a Java environment using an information hiding extraction of a decoder payload, where decoder payloads are created using specially written Java classes.
• Turning toFIG. 3, an exemplary server-side encoding embodiment is illustrated. In this example,content server100 hascontent320, which includes amedia file302 configured to be requested and played onmedia player301 residing onuser device130. When media file302 is initialized, audio is extracted from the media file and, if the audio is encoded (e.g., MP3 audio), subjected to audio decoding in304 to produceraw audio305. To encode the audio for beaconing, device ID, HHID anchor PPMID data is provided forfirst encoding306 the data into theraw audio305, using any suitable technique described above.
• After the first encoding, the audio data is then subjected to a second encoding to transform the audio into a suitable format (e.g., MP3) to produce fully encodedaudio308, which is subsequently transmitted tomedia player301 and beaconed toportable device140. Alternately, encodedaudio308 may be produced in advance and stored as part ofmedia file302. During the encoding process illustrated inFIG. 3, care must be taken to account for processing delays to ensure that the encoded audio is properly synchronized with any video content inmedia file302.
• The server-side encoding may be implemented under a number of different options. A first option would be to implement a pre-encoded beacon, where the encoder (306) would be configured to perform real-time encoding of the audio beacon based on the content being served to the users/panelists. The user device would be equipped with a software decoder as described above which is invoked when media is played. The pre-encoded beacon would establish a message link which could be used, along with an identifier from the capturingportable device140, in order to assign credit. The encoding shared library would preferably be resident at the content site (100) as part of the encoding engine. Such a configuration would allow the transcoding and encoding to be fit into the content site workflow.
• Another option for server-side encoding could include a pre-encoded data load, where the audio is encoded with a message that is based on the metadata or the assigned URL. This establishes a message link which can be used, along with an identifier from the capturingportable device140, in order to assign credit. The encoding shared library is preferably resident at the content site (100), as part of the encoding engine. Again, this configuration would allow the transcoding and encoding to be fit into the content site workflow.
• Yet another option for server-side encoding could include “on-the-fly” encoding. If a video is being streamed to a panelist, encoding may be inserted in the stream along with a transcoding object. The encoding may be used to encode the audio with a simple one of N beacon, and thepanelist user device130 would contain software decoding which is invoked when the video is played. This also establishes a message link which can be used, along with an identifier from the capturingportable device140, in order to assign credit. The encoding shared library is preferably resident at the content site (100), as part of the encoding engine. Under a preferred embodiment, an ActionScript would invoke the decoding along with a suitable transcoding object.
• FIG. 4 illustrates one embodiment for encoding media under aFlash Video platform410, where the content is preferably encoded in advance. As raw audio from a video file orother source400 is received, the audio is subjected towater mark encoding401, which may include techniques described above for the encoding. Once encoded, the audio is formatted as a Flash file using Adobe Tools402 such as FLV Creator and SWF Compiler. Once compiled, the file is further formatted using Flash-supported codecs (e.g., H.264, VP6, MPEG-4 ASP, Sorenson H.263) andcompression403 to produce a watermarked A/V stream or file404.
• FIG. 5 provides another alternate embodiment that illustrates client-side encoding and processing. In this example,user device130 requests media data. In response to the request, amedia file531 residing oncontent server100 is subsequently streamed to the device'sbrowser520 arranged on user'sworkspace510.Media player521 plays the streamed content and producesraw audio511. A client-side ActionScript notifiesbrowser522 andencoder522 to capture the raw audio on the device's sound mixer, or microphone (not shown), and to encode data using a suitable encoding technique described above. The encoding constructs the data for an independent audio beacon using the captured audio and other data (e.g., device ID, HHID, etc.) whereportable device140 picks up the beacon and forwards the data to an appropriate server for further processing and panel data evaluation.
• Similar to the server-side embodiment disclosed inFIG. 3, care must be taken in the software to account for processing delays in audio pickup and (CBET) encoding of the audio beacon. Preferably, synchronization between audio beacon playback and audio playback (specifically FLV playback) should be accounted for. In alternate embodiments, communication betweenmedia player521 andencoder522 could be through Actionscript interface APIs, such as “ExternalInterface”, which is an application programming interface that enables straightforward communication between ActionScript and a Flash Player container; for example, an HTML page with JavaScript, or a desktop application with Flash Player embedded, along withencoder application522. To get information on the container application, an ActionScript interface could be used to call code in the container application, including a web page or desktop application. Additionally, ActionScript code could be called from code in the container application. Also, a proxy could be created to simplify calling ActionScript code from the container application.
• For the panel-side encoding, a beacon embodiment may be enabled by having an encoding message being one from a relatively small set (e.g., 1 of 12), and where eachuser device130 is assigned a different message. Whenportable device140 detects the encoded message, it identifies theuser device130. Alternately, the encoding message may be a hash of the site and/or URL information gleaned from the metadata. When a panelistportable device140 detects and reports the encoded message, a reverse hash can be used to identify the site, where the hash could be resolved on one or more remote server (e.g., sever160).
• In addition to the encoding techniques described above in connection with media content, a simplified beaconing configuration may be arranged where the beacon operates as a complement to media data, independent of the media data, or providing a beacon where no specially encoded data exists. Referred to herein as a “twinkle,” the simplified beaconing comprises a constant amplitude acoustic signal or tone that is generated onuser device130. This acoustic tone is then automatically encoded, preferably with identification data (e.g., device ID, HHID and/or PPM ID) and a timestamp. The encoded acoustic tone would then be forwarded toportable device140 for processing and identification.
• The acoustic tone used for the twinkle is preferably embodied as a pre-recorded constant amplitude tone that is transmitted at predetermined times. The encoding is preferably performed using any of the techniques described above. Under one embodiment, the simplified beaconing process would only forward the encoded, pre-recorded tone, independently of any audio data being received. Thus, referring back toFIG. 1, it is possible thatuser device130 receives onlyother content114 fromcontent server110 in the form of text-based HTML. AsPC meter132 records browsing information, the encoded tone is transmitted toportable device140, where after further processing (seeFIG. 1B), the user identification data is merged intointernet measurement database141. It is understood, thatuser device140 may also receive audio data (encoded or unencoded) separately and in addition to other content. While the techniques described above would encode and forward audio data received, the simplified beacon (“twinkle”) would also transmit ID information toportable device140, which, in conjunction withPC meter132, would subsequently merge panelist data into a common database.
• In another exemplary embodiment,FIG. 6 illustratesaudio signal600 represented as a spectrum ofaudio610 over a period of time (e.g., 0.25 seconds), where the energy intervals vary with frequency between 1200 and 2200 Hz. Overlaid in black are discreet, narrowband code tones602 (e.g., CBET) opportunistically inserted into the audio using the principles of psychoacoustic masking. For encoded tones, one of which is illustrated as602 inFIG. 6, the energy of the inserted code tone varies with the level of audio, so more quiet portions of the frequency spectrum (e.g.,604) receive little encoding energy and compared to louder portions605), which get proportionally more.
• In contrast, the simplified encoding (“twinkle”)603 is encoded and inserted at constant levels across the frequency spectrum, where the levels are independent of the audio levels. This allows the simplified encoding to be pre-recorded, easily generated and capable of being reused accross various and/or different content. The simplified encoding could have the same message structure as the CBET encoding described above, utilizing a 10-tone symbol set. Alternately, other message structures are possible as well. As mentioned above, the twinkle may be transmitted automatically at regular intervals. Alternately the twinkle may be invoked by an ActionScript. If two players are utilized (i.e., one for the media, and one for the twinkle), the ActionScript could relay a beacon for the media fromuser device130 toportable device140, while simultaneously requesting a second (preferably invisible) Flash Player in theuser device130 to transmit the twinkle toportable device140. Under a preferred embodiment, the ActionScript should invoke both players at a common volume setting.
• In certain embodiments, it is advantageous to configure the audio beacon or “twinkle” on a device so that it is not intrusive and/or distracting to the user. Additionally, the beacon may be arranged to have audio characteristics that make it easier and more robust to encode. For example, simulated environmental sounds, such as power supply fans, air vent exhaust, crowd/audience noise, ocean waves and such may be used as the audio beacon sound in which encoded messages may be inserted. In another example, it is known that many computing devices, such as personal computers, phones, tablets and laptops, contain a pre-stored library of audio that is used to alert or notify users. By creating and storing sounds advantageously suited to encode data in the on-device sound library, a more robust beacon may be utilized. Furthermore, since the sound used for the beacon is known a priori, the encoding on the device may be simplified, since the time, frequency, masking and other encoding processes will be known.
• Turning toFIG. 7, an exemplary embodiment is illustrated whereuser device130 is configured with acontrol panel700 that allows the device to control various aspects, includingsound701 that includes aresident sound library720. In a preferred embodiment, a script, operating alone or as part of meter software (seeFIG. 1A, ref.132), may control aspects ofsound library720, such that specific sounds are used for audio beacon encoding. Of course, it is possible for a user to manually change these features either directly throughuser device130, or through a remote connection.
• Sound library720 comprises audio sounds (704-707) that are associated with one or more software applications700-701 and/or events702-703. “Events,” for the purposes ofFIG. 7, include notifications (e.g., receipt of email, social networking software status update, etc.) and alerts (e.g., start-up, shut-down, application error, etc.) that are made pursuant to the device's operating system and associated software running on the device. In this example, a first application (APP1)700 and a second application (APP2)701 are associated with respective sounds704-705 that may be triggered when the application is activated, or at predetermined times determined by the specific application. Applications700-701 may includemetering software132 discussed above in connection withFIG. 1A.
• In one embodiment, eachsound704,705 is associated with arespective application700,701. The sounds (704-705) are used for encoding messages to form respective beacons (708-709). As data for the audio beacon is being collected on user device130 (e.g., user device ID, web hash, etc.), the sound (e.g.,704) is copied, data is encoded into it and the encoded copy is stored in a buffer or other suitable memory. As further data for an audio beacon is collected, another copy of the original sound (704) is made, and the further data is encoded and stored into the buffer/memory. This process repeats for as long as necessary to form a string of encoded sounds. The sounds may be arranged sequentially or in other suitable formats. When anapplication700 triggers a sound, afirst beacon708 is audibly transmitted. When the next trigger occurs, the next beacon is audibly transmitted, and so on, until the buffer/memory is empty, or a predetermined amount of time has expired. In another embodiment, one application can control a plurality of sounds (704-705) and produce a plurality of encoded beacons (708-709).
• Continuing withFIG. 7, events702-703 may be associated with respective sounds706-707, similar to applications700-701 discussed above, to produce beacons710-711. This configuration may be particularly advantageous to beacon information when a user performs an act on the device, such as open/close a browser window, open/close a tab on a browser, open/close an application, etc. As data for the audio beacon is being collected (e.g., user device ID, web hash, etc.), the sound (e.g.,706) is copied, data is encoded into it and the encoded copy is stored in a buffer or other suitable memory. As further data for an audio beacon is collected, another copy of the original sound (706) is made, and the further data is encoded and stored into the buffer/memory. This process repeats for as long as necessary to form a string of encoded sounds. Again, the sounds may be arranged sequentially or in other suitable formats. When an event (702) is detected, afirst beacon710 is audibly transmitted. When the next event occurs, the next beacon is audibly transmitted, and so on, until the buffer/memory is empty, or a predetermined amount of time has expired.
• As mentioned previously, the sounds704-707 are preferably predetermined and may simulate an environmental sound so as not to be intrusive or distracting to the user. Additionally, the sound may be selected to contain audio characteristics (e.g., having high masking levels in critical frequency bands) that makes it conducive to robust audio encoding. By using a predetermined sound for the audio beacon encoding, designers can have more flexibility in audibly beaconing data. In addition to audio characteristics, the predetermined sounds may have different lengths as well. In an example where an application (700) controls multiple sounds, the sounds may be the same instance of one sound, but having different lengths (e.g., 5 sec., 10 sec., etc.). In cases where a device's (130) volume is lower, the application may default to a longer sound to increase the probability of the beacon code being detected. If the volume increases, thedevice130 can switch to a shorter sound. This configuration has the added benefit of ensuring that users maintain a sufficient volume on their device to avoid longer (and possibly more intrusive) beacons.
• Various embodiments disclosed herein provide devices, systems and methods for performing various functions using an audience measurement system that includes audio beaconing. Although specific embodiments are described herein, those skilled in the art recognize that other embodiments may be substituted for the specific embodiments shown to achieve the same purpose. As an example, although terms like “portable” are used to describe different components, it is understood that other, fixed, devices may perform the same or equivalent functions. Also, while specific communication protocols are mentioned in this document, one skilled in the art would appreciate that other protocols may be used or substituted. This application covers any adaptations or variations of the present invention. Therefore, the present invention is limited only by the claims and all available equivalents.

Claims (21)

1-18. (Canceled)

19. A computing device comprising:

memory including computer readable instructions; and

a processor to execute the instructions to at least:

select, based on a volume setting of the computing device, one of a plurality of sound signals stored at the computing device, the plurality of sound signals having different lengths;

encode a digital message in the selected one of the plurality of sound signals to form an encoded sound signal; and

cause the encoded sound signal to be audibly output from the computing device in response to a trigger.

20. The computing device ofclaim 19, wherein the processor is further to collect data to include in the digital message, the data to include at least one of an identifier associated the computing device or hash of a data associated with a web site accessed by the computing device.

21. The computing device ofclaim 19, wherein to select the one of the plurality of sound signals, the processor is to:

select a first one of the plurality of sound signals having a first length when the volume setting of the computing device is a first volume setting; and

select a second one of the plurality of sound signals having a second length that is longer than the first length when the volume setting of the computing device is a second volume setting that is lower than the first volume setting.

22. The computing device ofclaim 19, wherein the processor is further to:

store the encoded sound signal in a buffer including a plurality of encoded sound signals encoded with respective digital messages; and

cause respective ones of the plurality of encoded sound signals included in the buffer to be audibly output sequentially from the computing device in response to a sequence of triggers.

23. The computing device ofclaim 22, wherein the processor is to continue to cause respective ones of the plurality of encoded sound signals included in the buffer to be audibly output sequentially from the computing device in response to the sequence of triggers until at least one of the buffer is empty or an amount of time has expired.

24. The computing device ofclaim 19, wherein the plurality of sound signals is associated with an application to execute on the computing device, and the application is to trigger the processor to cause the encoded sound signal to be audibly output from the computing device.

25. The computing device ofclaim 19, wherein the plurality of sound signals is associated with an operating system event of the computing device, and occurrence of the operating system event is to trigger the processor to cause the encoded sound signal to be audibly output from the computing device.

26. A computer readable storage device comprising computer readable instructions that, when executed by a processor of a computing device, cause the processor to at least:

select, based on a volume setting of the computing device, one of a plurality of sound signals stored at the computing device, the plurality of sound signals having different lengths;

encode a digital message in the selected one of the plurality of sound signals to form an encoded sound signal; and

cause the encoded sound signal to be audibly output from the computing device in response to a trigger.

27. The storage device ofclaim 26, wherein the instructions further cause the processor to collect data to include in the digital message, the data to include at least one of an identifier associated the computing device or hash of a data associated with a web site accessed by the computing device.

28. The storage device ofclaim 26, wherein to select the one of the plurality of sound signals, the instructions cause the processor to:

select a first one of the plurality of sound signals having a first length when the volume setting of the computing device is a first volume setting; and

select a second one of the plurality of sound signals having a second length that is longer than the first length when the volume setting of the computing device is a second volume setting that is lower than the first volume setting.

29. The storage device ofclaim 26, wherein the instructions further cause the processor to:

store the encoded sound signal in a buffer including a plurality of encoded sound signals encoded with respective digital messages; and

cause respective ones of the plurality of encoded sound signals included in the buffer to be audibly output sequentially from the computing device in response to a sequence of triggers.

30. The storage device ofclaim 29, wherein the instructions cause the processor to continue to cause respective ones of the plurality of encoded sound signals included in the buffer to be audibly output sequentially from the computing device in response to the sequence of triggers until at least one of the buffer is empty or an amount of time has expired.

31. The storage device ofclaim 26, wherein the plurality of sound signals is associated with an application to execute on the computing device, and the application is to trigger the processor to cause the encoded sound signal to be audibly output from the computing device.

32. The storage device ofclaim 26, wherein the plurality of sound signals is associated with an operating system event of the computing device, and occurrence of the operating system event is to trigger the processor to cause the encoded sound signal to be audibly output from the computing device.

33. A method for a computing device, the method comprising:

selecting, by executing an instruction with a processor of the computing device and based on a volume setting of the computing device, one of a plurality of sound signals stored at the computing device, the plurality of sound signals having different lengths;

encoding, by executing an instruction with a processor, a digital message in the selected one of the plurality of sound signals to form an encoded sound signal; and

audibly outputting the encoded sound signal from the computing device in response to a trigger.

34. The method ofclaim 33, further including collecting data to include in the digital message, the data including at least one of an identifier associated the computing device or hash of a data associated with a web site accessed by the computing device.

35. The method ofclaim 33, wherein the selecting of the one of the plurality of sound signals includes:

selecting a first one of the plurality of sound signals having a first length when the volume setting of the computing device is a first volume setting; and

selecting a second one of the plurality of sound signals having a second length that is longer than the first length when the volume setting of the computing device is a second volume setting that is lower than the first volume setting.

36. The method ofclaim 33, further including:

storing the encoded sound signal in a buffer including a plurality of encoded sound signals encoded with respective digital messages; and

audibly outputting respective ones of the plurality of encoded sound signals from the buffer sequentially in response to a sequence of triggers until at least one of the buffer is empty or an amount of time has expired.

37. The method ofclaim 33, wherein the plurality of sound signals is associated with an application to execute on the computing device, and the application is to trigger the encoded sound signal to be audibly output from the computing device.

38. The method ofclaim 33, wherein the plurality of sound signals is associated with an operating system event of the computing device, and occurrence of the operating system event is to trigger the processor to cause the encoded sound signal to be audibly output from the computing device.

Images