Invention content
Technical problem
Incidentally, in the U.S., due to popularizing for CATV, IPTV etc., TV receivers are arranged to the display of external equipmentOutput device, the external equipment are, for example, the set-top box of CATV, IPTV as described above etc..
Therefore, user, which has few opportunities, operates the remote controler of TV receivers, and often operates the remote controler of external equipment.CauseThis, it is possible to, for the ease of use of user, in the content reproducing device as the remote controller operation by TV receiversTV receivers in the operation of associated application that starts may be unfavorable.
Accordingly, it is considered to the convenience of user, generates and generated so that the second screen apparatus is shown and justThe request of the relevant metadata of content or execution associated application watched on content reproducing device, the second screen apparatus are differentIn the content reproducing device (TV receivers etc.) for being integrated with ACR clients.
Here, as the second screen apparatus, for example, panel type terminal, tablet PC, smart phone etc. can be listed.
In addition, in recent years, can be set using the scheduled application program started in the second screen apparatus to operate outsideSecond screen apparatus of standby type (i.e., it is possible to type of the remote controler as external equipment) is obtainable.Therefore, fromFor the operation input device of content reproducing device and external equipment from the perspective of integrator on the second screen apparatus,It is expected that realizing above-mentioned request.
In addition, mainly in the U.S., existing trend is that the screen size of TV receivers as content reproducing device etc. becomesAt more than 50 to 60 inches, and on the increased screen of size show and watch simultaneously the use of mutually different a plurality of contentForm starts gradually universal.Therefore, it is necessary to also consider to build in ACR clients with this large scale screen and with secondUse form in associated TV receivers of screen apparatus etc..In the following description, there is large scale screen and can be sameWhen show that TV receivers of a plurality of different content etc. are referred to as split screen device.
Fig. 1 be assuming that by the interior split screen device for having ACR clients and the second screen apparatus it is associated with each other withThe schematic diagram in the space in the case of being serviced using ACR and be arranged in these devices in family of user etc..
In Fig. 1, split screen device 1-1 is installed in the front of user, and split screen device 1-2 is mountedIn left side wall.Second screen apparatus 3 is disposed near user.
Multiple (in the case of figure 1, three) windows can be set on split screen device 1-1, and can beA plurality of different content (can be mutually the same) is shown in these windows simultaneously.In the following description, for example, being shown in window AThe content shown is referred to as content A.This is also applied similarly to the other a plurality of contents shown in other windows.
Split screen device 1-1 can be serviced to the second screen apparatus 3 by using ACR and be transmitted ACR responses, the ACRResponse includes the recognition result etc. of a plurality of content shown in window.
In addition, split screen device 1-2 is similar to split screen device 1-1.However, in Fig. 1, split screenDevice 1-2 is in that only window X is set on its screen and content X is displayed on state in window X.
Second screen apparatus 3 can show the window with split screen device 1-1 or split screen device 1-2 simultaneouslyMiddle display a plurality of content corresponding metadata and the associated application that is started according to the progress of content.
In the case of figure 1, the second screen apparatus 3 is in while showing the associated application A of the content A for window A, usesIn the state of the metadata X of the content X of another associated application C and window X of the content C of window C.
Assuming that in the case of use form shown in Fig. 1, it is possible to, user possibly can not quickly grasp the second screenThe a plurality of information (video and metadata of associated application) shown on the screen of proscenia 3 and split screen device 1-1 and 1-2Window in the correspondence of a plurality of content that shows.
Therefore, it is necessary to make user quickly grasps can be in the window of split screen device 1-1 and 1-2 while aobviousBetween a plurality of content shown and a plurality of information (video and metadata of associated application) that can be shown on the second screen apparatus 3Correspondence invention.
Present disclosure is made according to this situation as described above, and user is quickly grasped canWith the correspondence simultaneously between a plurality of content shown and the information shown respectively.
Solution to the problem
According to present disclosure in a first aspect, provide a kind of information processing equipment, described information processing equipment and endEnd equipment cooperates to constitute domestic network system, and described information processing equipment includes：Frame perceptual property setting section, is configured toFrame perceptual property is set with the window of the content of display reproduction on the screen to being arranged；Open part, is configured to set to terminalThe order of the notice of the standby open frame perceptual property for being set from premises equipment requests to window；And notification section, by structureIt causes, in response to calling the order from the terminal device, the frame perceptual property of the window to be notified to the terminal device.
The display control section can show setting on the screen multiple windows of a plurality of different content are setDifferent frame perceptual property and the open part can be disclosed to terminal device for from the multiple window of premises equipment requestsThe order of the notice of the frame perceptual property of mouth.
The frame perceptual property setting section may be in response to order the request sent out from the terminal device using describedChange the frame perceptual property of window.
The open part can the direct order described in the defined device description of extension UPnP URL, with toThe terminal device discloses the URL of the order.
The order can be WebAPI.
The open part can carry out multicast distribution on the home network to notification message, and in response to being based onThe access from the terminal device for stating notification message discloses described device description to the terminal device.
The disclosure, which partly may be in response to search for the terminal device from the M of the terminal device, discloses described deviceDescription.
According to present disclosure in a first aspect, described information processing equipment can also include：Identify request processing section,It is configured to the signed data that the extraction from the content of reproduction represents the characteristic of the content, is generated described including at least being extractedThe inquiry of signed data and the inquiry is sent to identification server, and received in response to the inquiry from the identificationThe response of the recognition result of content that is that server is sent back and representing the reproduction, wherein the open part can also be to endEnd equipment discloses the order for enabling the identification request processing section to work from terminal device.
According to present disclosure in a first aspect, providing a kind of information processing method for information processing equipment, instituteIt states information processing equipment to be cooperated with terminal device to constitute domestic network system, the method is held by described information processing equipmentRow, the method includes：Frame perceptual property setting procedure sets frame with the window of the content of display reproduction on the screen to be arrangedPerceptual property；Open step discloses leading to for the frame perceptual property for set from premises equipment requests to window to terminal deviceThe order known；And notifying process, in response to calling the order from the terminal device, described in terminal device noticeThe frame perceptual property of window.
According to present disclosure in a first aspect, provide a kind of program, cooperated with terminal device to constitute for makingThe computer of domestic network system is used as：Frame perceptual property setting section, is constructed and is arranged in pairs on the screen with display reproductionContent window set frame perceptual property；Open part is configured to disclose for from premises equipment requests to terminal deviceOrder to the notice of the frame perceptual property of window setting；And notification section, it is configured in response to from the terminal deviceThe order is called, the frame perceptual property of the window is notified to the terminal device.
In present disclosure in a first aspect, frame vision is arranged with the window of the content of display reproduction on the screen to being arrangedAttribute, and the order of the notice of the frame perceptual property for window to be arranged from premises equipment requests is disclosed to terminal device.In addition, in response to from terminal device call instruction, to the frame perceptual property of terminal device notification window.
According to the second aspect of the present disclosure, a kind of cooperated with content display apparatus to constitute home network system is providedThe information processing equipment of system, the content display apparatus show content, described information processing in window on the screen is arrangedEquipment includes：Order fetching portion is configured to obtain for asking to being arranged in the content from the content display apparatusIt shows the order of the notice for the frame perceptual property that the window on the screen of equipment with the content of display reproduction is set, and calls instituteOrder is stated to receive the notice of the frame perceptual property；Associated application execution part, being configured to obtain and execute will basisThe associated application that the progress of the content shown in window on the screen of the content display apparatus executes is set；And frame is aobviousShow control section, is configured to identical as the frame perceptual property of notice to be set by the frame for the image that the associated application is shownFrame perceptual property.
The order fetching portion can be obtained disclosed in by the content display apparatus and be extended as defined in UPnPThe URL of the order directly described in device description.
The order can be WebAPI.
The order fetching portion can based on by the content display apparatus on the home network multicast distribution it is logicalKnow that message obtains described device description from the content display apparatus.
The order fetching portion can be obtained search out by searched for by M from described in content display apparatus supplyDevice describes.
The content display apparatus may include：It identifies request processing section, is configured to extract from the content of reproductionThe signed data of the characteristic of the content is represented, the inquiry including at least extracted signed data is generated and sends out the inquiryIt is sent to identification server, and receives being sent back from the identification server in response to the inquiry and represents the reproductionContent recognition result response.The order fetching portion can also be obtained for so that described from the content display apparatusThe order that the identification request processing section of content display apparatus can work, and by calling the order to receive from instituteState content display apparatus transmission response and the associated application execution part can include based on the response receivedAnd the corresponding associated metadata of content with identification, obtain and execute the associated application.
According to the second aspect of the present disclosure, it provides a kind of for being cooperated with content display apparatus to constitute home networkThe information processing method of the information processing equipment of network system, the content display apparatus are shown in window on the screen is arrangedContent, the method are executed by described information processing equipment, the method includes：Obtaining step is obtained for from the contentThe frame that display device request sets the window that the content on the screen of the content display apparatus with display reproduction is arranged regardsFeel the order of the notice of attribute；Receiving step calls the order to receive the notice of the frame perceptual property；Associated application is heldRow step, obtain and execute will according to the content shown in the window being arranged on the screen of the content display apparatus intoSpend the associated application executed；And frame display control step, to be set and be led to by the frame for the image that the associated application is shownThe identical frame perceptual property of frame perceptual property known.
According to the second aspect of the present disclosure, provide a kind of program, for so that cooperate with content display apparatus withThe computer for constituting domestic network system is used as with lower part, and the content display apparatus is shown in window on the screen is arrangedShow content：Order fetching portion is configured to obtain for aobvious in the content to setting from content display apparatus requestShow the order of the notice of the frame perceptual property of the window setting of the content on the screen of equipment with display reproduction, and described in callingOrder is to receive the notice of the frame perceptual property；Associated application execution part, being configured to obtain and execute will be according to settingSet the associated application that the progress of the content shown in the window on the screen of the content display apparatus executes；And frame is shownControl section is configured to identical with the frame perceptual property of notice to be set by the frame for the image that the associated application is shownFrame perceptual property.
In the second aspect of the present disclosure, it is obtained from content display apparatus for asking to being arranged in content display apparatusScreen on be arranged with the window of the content of display reproduction frame perceptual property notice order, and call the order to connectReceive the notice of frame perceptual property.Obtain and execute will be according to being shown in the window being arranged on the screen of content display apparatus inThe associated application that the progress of appearance executes, and to the image of associated application to be shown frame setting and the frame perceptual property phase that notifiesSame frame perceptual property.
According to the third aspect of present disclosure, a kind of information processing method for domestic network system, institute are providedIt includes content display apparatus and terminal device to state domestic network system, and described information processing method includes：It is shown by the contentThe frame perceptual property setting procedure that equipment carries out sets frame vision category with the window of the content of display reproduction on the screen to being arrangedProperty；The open step carried out by the content display apparatus is disclosed to terminal device for being set from premises equipment requests to windowThe order of the notice of fixed frame perceptual property；The obtaining step carried out by the terminal device obtains the disclosed order；ByThe notifying process that the content display apparatus carries out is set in response to calling the order by the terminal device to the terminalThe standby frame perceptual property for notifying the window；Step is executed by the associated application that the terminal device carries out, obtains and executesThe associated application to be executed according to the progress of the content shown in the window being arranged on the screen of the content display apparatus；WithAnd the frame display control step carried out by the terminal device, to be set and be led to by the frame for the image that the associated application is shownThe identical frame perceptual property of frame perceptual property known.
In the third aspect of present disclosure, by content display apparatus, to being arranged on the screen with display reproductionFrame perceptual property is arranged in the window of content, and the frame disclosed to terminal device for window to be arranged from premises equipment requests regardsFeel the order of the notice of attribute.Meanwhile by terminal device, obtaining disclosed order.In addition, by content display apparatus, ringYing Yu is by terminal device call instruction, to the frame perceptual property of terminal device notification window.Then, it by terminal device, obtainsAnd execute the associated application to be executed according to the progress of the content shown in the window being arranged on the screen of content display apparatus.Therefore, frame perceptual property identical with the frame perceptual property of notice is arranged to the frame of the image of associated application to be shown.
Beneficial effects of the present invention
Using present disclosure in a first aspect, the frame that the window for showing content is arranged can be notified to regard to terminal deviceFeel attribute.
Using the second aspect of present disclosure, the associated application that can be executed by the progress according to the content is rightFor showing that it is identical with to showing the frame perceptual property that the window of content is arranged on content display apparatus that the frame of image is arrangedFrame perceptual property.
It, can be by the association that is executed according to the progress of content by terminal device using the third aspect of present disclosureUsing public frame vision is arranged in both frames to showing the window of the content on content display apparatus and for showing imageAttribute.
Specific implementation mode
In the following, the optimal mode being described in detail with reference to the drawings for executing present disclosure (hereinafter, is referred to as implementingExample).
[summaries of ACR system using services]
Fig. 2 shows the summaries of the ACR system using services of the embodiment as present disclosure.ACR system using servicesThe second screen apparatus 50 including split screen device 40-1 and 40-2 and with built-in ACR clients, and make theseDevice is associated with each other to use ACR to service.
In fig. 2, split screen device 40-1 is installed in front of user, and split screen device 40-2 is mountedIn left side wall.Near user, the second screen apparatus 50 is arranged.
Multiple (in the case of figure 2, three windows) can be arranged on the screen of split screen device 40-1, andMutually different a plurality of content (can be mutually the same) can show simultaneously in the window.In the following description, for example, windowThe content shown in A is referred to as content A.This is also applied similarly to the other a plurality of contents shown in other windows.
In addition, split screen device 40-1, which can be serviced by using ACR to the second screen apparatus 50, transmits ACR soundIt answers, ACR responses include the recognition result etc. of a plurality of content shown in window.
It can visually (be wrapped with other windows in addition, split screen device 40-1 can be provided to the frame of each windowInclude the window of split screen device 40-2) the attribute (hereinafter, being referred to as frame perceptual property) that distinguishes of frame.
As frame perceptual property, can list can be by color, the thickness for the frame line that CSS (cascading style sheets) sets and changesDegree and type, the depth of Three-dimensional Display etc..In the case of figure 2, the frame line of each window is configured to and the frame of other windowsRow is different, and the frame line of window A is configured to alternately chain-dotted line；The frame line of window B is configured to solid line；And window CFrame line is configured to break.
Split screen device 40-2 is similar to split screen device 40-1.However, in fig. 2, split screen dressSetting 40-2, window X is set on its screen and the frame line of window X is configured to alternately double dot dash line to show in only havingIn the state of content X.
In the second screen apparatus 50, with a plurality of content pair shown in the window of split screen device 40-1 and 40-2The associated application of the metadata answered and the progress for following content can be activated and show.
In addition, in the second screen apparatus 50, can to every information to be shown (associated application corresponding with content orThe screen picture of metadata) frame is provided, and the frame vision category with the frame for the window for wherein showing corresponding content is set to the frameThe similar frame perceptual property of property.
In particular, in the case of figure 2, the second screen apparatus 50 is in following state：Simultaneously, with window A'sThe corresponding associated application A of content A are shown in alternately dashdotted frame；Associated application C corresponding with the content C of window C is shownIn the frame of broken string；And metadata corresponding with the content X of window X is shown in the alternately frame of double dot dash line.It should be noted thatIn addition, not being to show simultaneously in the case of switchably show in this plurality of information, it can be arranged to frame and wherein showThe similar frame perceptual property of the frame perceptual property of the frame of the window of corresponding content.
[the construction examples of ACR system using services]
Fig. 3 shows the construction example of the ACR system using services of the embodiment of present disclosure.
The ACR system using services 10 identification (such as, has big ruler in split screen device 40-1 and 40- 2 simultaneouslyVery little screen and presented and the TV receivers of the relevant metadata of a plurality of content to user by the second screen apparatus 50) screenOne or more content of upper display, or the progress of content is followed to execute associated application.In the following description, it is not needingIt is therein to be each referred to only as split screen in the case that split screen device 40-1 and 40-2 are distinguished independent of one anotherDevice 40.
Here, content is not limited to the program broadcasted.In particular, all AV contents are (all such as (e.g.) to be set by outsideStandby (video recorder such as, being connect with split screen device 40) records and the program reproduced, the film reproduced from encapsulation mediumDeng video and pass through the motion picture and music of the downloads such as internet) be included in content.In addition, be inserted into program itBetween or the commercial advertisement (hereinafter, being referred to as CM) of program midway be included in content.
It in order to identify content, is serviced using ACR, in ACR services, will periodically be extracted in the content fromAnd the characteristic that represents content signed data (for example, watermark or fingerprint) and a plurality of content in the database registered in advanceSigned data be compared, to identify content.
ACR system using services 10 by supplying content and associated application corresponding with the content etc. side supplier beThe domestic network system 30 of the user side of associated application is perhaps used to constitute in system 20 and viewing.
Supply method, system 20 and domestic network system 30 are connected to each other by broadcasting network 11 and bidirectional communication network 12.ThisIn, broadcasting network 11 includes TV broadcasting networks, catv network, IPTV networks etc..Bidirectional communication network 12 is using internet as generationThe IP communication networks of table.
It should be noted that in figure 3, belonging to the split screen device 40-2 and supply method, system of domestic network system 30The circuit by broadcasting network 11 and bidirectional communication network 12 between 20 is omitted.
[to supplying the description of method, system 20]
Method, system 20 is supplied by content server 21, meta data server 22, application server 23 and ACR servers 24It constitutes.
Content server 21 plays the content of such as program or CM by broadcasting network 11.In addition, content server 21 suppliesThe content of ACR servers 24 should be broadcast to.It should be noted that the replacement as supplying content, in advance out of contents extractionACR servers 24 can be supplied to by holding identifier and signed data.
Metadata corresponding with the content broadcasted from content server 21 is supplied to ACR servers by meta data server 2224.Metadata includes the associated application that executed according to the progress of content with the relevant various types of information of content, representativeIdentifier, the data etc. to be used in associated application.
Application server 23 passes through two-way communication according to the request of the second screen apparatus 50 from domestic network system 30The data that the supply of network 12 is applied and to be used in this application.
ACR servers 24 generate ACR reference datas, in ACR reference datas, content designator, application identifier in advanceDeng with the predetermined sampling period from each content that can be watched on the split screen device 40 of domestic network system 30The signed data of extraction is harmonious, to form database and keep the database.ACR servers 24 are based on ACR reference numbersAccording to database, inquired in response to the ACR that any of ACR clients 42A to 42C from split screen device 40 is sentIt is responded to generate ACR, and sends ACR responses.
It should be noted that constituting content server 21, meta data server 22, the application server 23 of supply method, system 20A place can be arranged in by centralized system with ACR servers 24, or can be arranged apart by dispersing mode.Alternatively,Can suitably in combine component some.Furthermore it is possible to provide multiple servers for each of component server.
[description to domestic network system 30]
Domestic network system 30 is by split screen device 40 (such as, the TV receivers with large scale screen) and secondScreen apparatus 50 (such as, panel type terminal, tablet PC or smart phone) is constituted.Split screen device 40 and the second screen dress50 are set for example to be connected to each other by Wireless LAN.
It should be noted that multiple this split screen devices 40 and multiple this second screen apparatus 50 can be arrangedIn domestic network system 30.
Split screen device 40 provide on its screen multiple windows and in the window at the same show it is a plurality of difference inHold, to be watched by user.Split screen device 40 includes content renderer 41, ACR clients 42, (general be to insert i.e. to UPnPWith) server 43, local Web server 44 and window manager 45.Be integrated with can be by split screen device 40This content renderer 41 and this ACR clients 42 of the equal quantity of number of windows of setting.
In this example, it is assumed that split screen device 40 can provide three windows A, B and C on its screen.?Under this situation, it is assumed that correspondingly integrate there are three content renderer 41A, 41B and 41C and three ACR clients 42A, 42B and42C.The quantity for the window that can be arranged is not limited to three.In the following description, need not be by content renderer 41A, 41BIt is therein to be each called content renderer 41 for short in the case of being distinguished independently of each other with 41C.This is also applied similarly toACR clients 42A, 42B and 42C.
Content renderer 41A is obtained and is shunted the institute to be shown in the window A on the screen of split screen device 40There is an a plurality of content, and by this plurality of Content supply to ACR clients 42A.In particular, content renderer 41A is not only obtainedAnd the content played from content server 21 is shunted, and obtain and be diverted through the various input terminals of split screen device 40The content of sub (HDMI (registered trademark) terminal etc.) input, and by the Content supply of acquisition to ACR clients 42A.This also classAs be applied to content renderer 41B and 41C.
ACR clients 42A is input to from from content renderer 41A in ACR clients 42A with the scheduled sampling periodExtraction represents the signed data of content character in appearance.Here, as the extracting method of signed data, using with by supply method, systemThe extracting method that 20 ACR servers 24 use identical extracting method (described in hereafter).
It is inquired in addition, ACR clients 42A generates the ACR including extracted signed data, and passes through bi-directional communication networkACR inquiries are sent to the ACR servers 24 of supply method, system 20 by network 12.In addition, ACR clients 42A is received and is kept loudThe ACR that Ying Yucong ACR clients 42A is sent inquires and is sent to from ACR servers 24 the ACR responses of ACR clients 42A.This is also applied similarly to ACR clients 42B and 42C.
UPnP servers 43 generally to construct domestic network system 30 together with split screen device 40 anotherDevice (in the present case, the second screen apparatus 50) notice is integrated with UPnP servers 43 split screens of itself for callingThe WebAPI's (application programming interfaces) of the function (in the present case, being serviced using ACR by ACR clients 42) of device 40 is thinSection.In particular, UPnP servers 43 redefine agreement based on the basic agreement that general UPnP servers provide, and useDefined agreement executes universal notice (being hereinafter described in detail).
Local Web server 44 so that constitute other devices of domestic network system 30 with split screen device 40 together(in the present case, the second screen apparatus 50) uses the split screen device 40 for being integrated with local Web server 44 itselfFunction (in the present case, is serviced by ACR clients 42 using ACR).In particular, local Web server 44 is in response to callingThe fact that the WebAPI learnt from UPnP servers 43 is to use ACR to service controls ACR clients 42, and will be taken from ACRThe ACR responses that business device 24 is sent back are transmitted to the second screen apparatus 50.
Window manager 45 controls the display of the frame for the window being arranged on the screen of split screen device 40.
In the present embodiment, the second screen apparatus 50 is mainly watched on split screen device 40 to user's presentationThe metadata of content, or associated application is executed according to the progress of content.Second screen apparatus 50 include multiple ACR using 51,Associated application manager 52 and window manager 53, the wherein quantity of associated application manager 52 apply 51 quantity phase with ACRDeng.
In this example, it is assumed that the second screen apparatus 50 is associated with including three ACR applications 51a, 51b with 51c and threeApplication manager 52a, 52b and 52c.However, their quantity is not limited to three.In the following description, need not be by ACRIn the case of individually being distinguished using 51a, 51b and 51c, each ACR that is referred to only as therein applies 51.This is also similarly answeredFor associated application manager 52a, 52b and 52c.
ACR is pre- installed appropriately using 51 in the second screen apparatus 50, and when with split screen device 40 at being associated withStart ACR when being serviced using ACR to relationship and applies 51.ACR is obtained from UPnP servers 43 using 51 and is used ACR for readingThe WebAPI of the function of service, and WebAPI is called to be received by ACR clients 42 by the acquisition of local Web server 44From ACR servers 24 ACR response.In addition, the ACR responses that ACR is obtained using 51 analyses, and analysis result is led toKnow to associated application manager 52.
First number that ACR responses include is presented to user using 51 notice according to from ACR in associated application manager 52According to, or obtained from application server 23 by bidirectional communication network 12 and execute and just seen on split screen device 40The corresponding associated application of content seen.It should be noted that Web browser is used as associated application manager 52.
The frame for the information that window manager 53 is shown display by each associated application manager 52 is executed and is wherein shownThe identical setting of setting of the frame perceptual property of the window of the split screen device 40 of corresponding content.
[the detailed configuration example of ACR servers 24]
Fig. 4 shows the detailed configuration example for the ACR servers 24 that supply method, system 20 includes.
ACR servers 24 are by content obtaining part 61, signature extraction part 62, ACR databases (DB) 63, ACR response productionsFirst portion 64 and communications portion 65 are constituted.
Content obtaining part 61 is scheduling to by content server 21 through broadcasting network 11 from equal obtain of content server 21The content broadcasted or each content that can be watched on split screen device 40, and by the Content supply of acquisitionPart 62 is extracted to signature.Signature extraction part 62 is using scheduled extracting method with the scheduled sampling period from from content obtainingPart 61 is supplied to the signed data that extraction in the content of signature extraction part 62 represents content character, and signed data is defeatedGo out to ACR databases 63.
As for the method that signed data is extracted in signature extraction part 62, existing any means can be applied.For example, can be withExtraction is previously embedded in the watermark in content and watermark is used as signed data, or can calculate fingerprint and use fingerprintAs signed data.
If fingerprint is used as signed data, even if the resolution ratio of content, aspect ratio, bit rate, coded format are turnedIt changes, identical signed data can also be obtained before conversion and later.Therefore, it can enhance and work as based in signed data executionThe recognition accuracy when identification of appearance.
It should be noted that in the case where being related to multiple ACR servers 24, each ACR servers 24 need not use phaseThe same method that signed data is extracted by signature extraction part 62, but each of ACR servers 24 can be directed to and used and appointedThe extracting method of meaning.
ACR databases 63 generate and store ACR reference datas, and in ACR reference datas, representative is therefrom extracted each labelContent designator of content of extraction source of name data etc. is associated with each signed data inputted from signature extraction part 62.CauseThis, the signed data extracted from content is converted into database.Hereinafter, the data knot of ACR reference datas is described with reference to Fig. 4Structure.
ACR responses generate part 64 with reference to ACR databases 63 to identify that being sent to ACR responses from ACR clients 42 generatesThe content of the extraction source for the signed data that the ACR inquiries of part 64 include.In addition, ACR responses, which generate part 64, generates ACR soundIt answers, ACR responses include content designator, the associated metadata etc. as recognition result.Hereinafter, it is described with reference to Fig. 8The data structure of ACR responses.The ACR that the reception of communications portion 65 is sent to communications portion 65 from ACR clients 42 is inquired and willACR inquiries are output to ACR responses and generate part 64, and will respond the ACR responses that generation part 64 generates by ACR and be sent toACR clients 42.
[the detailed configuration example of ACR clients 42]
Fig. 5 shows the example of the detailed configuration for the ACR clients 42 being integrated in split screen device 40.
ACR clients 42 generate part 72, communications portion 73 and ACR responses by signature extraction part 71, ACR inquiries and keepPart 74 is constituted.
It is extracted from from the content that content renderer 41 shunts and supplies with the scheduled sampling period signature extraction part 71The signed data of content character is represented, and signed data is output to ACR inquiries and generates part 72.It should be noted that signatureIt extracts part 71 and extracts the method for signed data and the signature extraction extraction of part 62 signed data of ACR servers 24 to be usedMethod be shared.
ACR inquiries generate part 72 and generate ACR inquiries, and ACR inquiries include the signature extracted by signature extraction part 71Data, and ACR inquiries are for asking ACR servers 24 to identify content.Hereinafter, the number of ACR inquiries is described with reference to Fig. 7According to structure.
Communications portion 73 will be inquired by ACR to be generated the ACR inquiries that part 72 generate and is sent to ACR servers 24, receive fromACR servers 24 are sent to the ACR responses of communications portion 73, and the ACR received responses are output to ACR responses and are keptPart 74.ACR response holding parts 74 keep being responded by the ACR that communications portion 73 receives.It should be noted that passing through localThe ACR responses of holding are transmitted to the second screen apparatus 50 by Web server 44.
[data structure of ACR reference datas]
Fig. 6 shows to be formed as the data knot of the ACR reference datas of the database in the ACR databases 63 of ACR servers 24Structure.
In ACR reference datas 100, content designator 102, recovery time 103 and associated metadata 104 with predetermined weekPhase is associated with from each signed data 101 of contents extraction.
Part 62 is extracted from contents extraction signed data 101 by signature.Content designator 102 is by procuration dataThe information of the content of 101 extraction source.The instruction of recovery time 103 from content when extracting signed data 101 by content identificationThe progress timing of the content that symbol 102 represents, and with for example from this content foremost period for passing indicate.
Relevant metadata 104 includes the metadata of the content indicated by content designator 102, associated application identifierIt (uniquely indicating will be according to the information for the associated application that the timing of the progress of the content indicated by the recovery time 103 executes) and will be byThe various data that performed associated application uses.
It should be noted that associated metadata 104 can have the progress timing according to the content indicated by the recovery time 103And different content.
[data structure of ACR inquiries]
Fig. 7 shows to generate the data structure for the ACR inquiries that part 72 generates by the ACR inquiries of ACR clients 42.
ACR inquiries 110 include ACR query identifiers 111, signature type identifier 112, service identifier 113, signatureData 114, local time stamp 115, source address 116 and signature 117.
ACR query identifiers 111 are the information for uniquely specifying ACR inquiries 110.Signature type identifier 112 isInformation for the specified extracting method used when extracting signed data 114.Service identifier 113 is for there are multipleSelect ACR inquiries 110 by one in the ACR servers 24 being sent to obtain ACR responses in the case of ACR servers 24Information.
Part 71 is extracted by the signature of ACR clients 42, and signed data 114 is extracted from content.Local time stamp 115Indicate the timing of the extraction signed data 114 from content, and indicated by the local system clock of split screen device 40Time indicate.
Source address 116 is the address for the ACR clients 42 that ACR inquiries 110 are initiated in bidirectional communication network 12, and is used asIt represents by ACR servers 24 in response to the specified information of the transmission of the ACR responses of 110 generation of ACR inquiries.Signature 117 prevents pseudo-Make ACR inquiries 110.It should be noted that ACR inquiries 110 can be sent after ACR inquiries 110 are completely encoded.
[data structure of ACR responses]
Fig. 8 shows to respond the data structure for generating the ACR responses that part 64 generates by ACR.
Data structure shown in ACR responses 120 A or B with Fig. 8.The A of Fig. 8 shows successfully identifying ACR(that is, in the user of split screen device 40 in the case of the content of the extraction source for the signed data 114 that inquiry 110 includesIn the case of successfully identifying the content watched) ACR response 120 data structure.The B of Fig. 8 show not atWork(identify content in the case of ACR response 120 data structure.
ACR responses in the case where the user of split screen device 40 successfully identifies the content watchedIn 120 (A of Fig. 8), including ACR query identifiers 121, signature type identifier 122, service identifier 123, content designator124, recovery time 125, local time stamp 126, associated metadata 127, source address 128 and signature 129.
Meanwhile in the case where the user of split screen device 40 does not identify the content watched successfullyACR is responded in 120 (B of Fig. 8), including ACR query identifiers 121, service identifier 122, source address 128, signature 129 and knowledgeOther failure flags 130.
ACR query identifiers 121, signature type identifier 122, service identifier 123 and local time stamp 126 are for referring toFixed request sends the ACR inquiries 110 of ACR responses 120, and quote and describe ACR inquire 110 ACR query identifiers 111,Signature type identifier 112, service identifier 113 and local time stamp 115.
Content designator 124, recovery time 125 and associated metadata 127 represent the recognition result of content, and reference is simultaneouslyContent designator 102, recovery time 103 and the associated metadata 104 of corresponding ACR reference datas 100 are described.
Source address 128 is the address for the ACR servers 24 that ACR responses 120 are sent in bidirectional communication network 12.Signature 129For preventing to forge ACR responses 120.It should be noted that ACR responses 120 can be encrypted and be sent to prevent ACR completelyThe forgery of response 120.
Recognition failures mark 130 is that ACR servers 24 are wrapped in ACR inquiries 110 of the identification from ACR clients 42Fail when content (that is, content that the user of split screen device 40 is watching) of the extraction source of the signed data 114 includedMark.
[processing of the domestic network system 30 of ACR system using services 10]
Now, two when 50 acquisition device of the second screen apparatus in description home network system 30 describes do not exist togetherThe description of reason, wherein device is the relevant information of function that can be used with split screen device 40.
It should be noted that the function that the split screen device 40 in the present embodiment can use is indicated by ACR clients42 are serviced using ACR, and frame perceptual property is applied to each window on screen by window manager 45.
Fig. 9 is the flow chart for showing the processing under following situations：It has been opened in the second screen apparatus 50 using 51 in ACRIn the state of dynamic, it (also includes determining for the startup of split screen device 40 that split screen device 40, which is connected to network system 30,When).
In step sl, the UPnP servers 43 of split screen device 40 will be defined logical by UPnP presented belowKnow that message multicast is distributed to domestic network system 30.Notification message is transmitted to domestic network system 30 using UDP/IP, andIn the present case notification message is received using 51 by the ACR of the second screen apparatus 50.
Notify * HTTP/1.1
Position " http://192.168.0.4:55500"
In " position " head of notification message, the acquisition destination of the device description of description split screen device 40Location.Therefore, in step s 11, the ACR of the second screen apparatus 50 is obtained using HTTP GET from destination-address is obtained using 51Device is taken to describe.
By above-mentioned processing, the ACR of the second screen apparatus 50 applies 51 dresses that can obtain split screen device 40Set description.
Figure 10 is the state for showing then to have been coupled to network system 30 in split screen device 40 and startUnder in the second screen apparatus 50 start ACR apply 51 when processing flow chart.
In this case, since the UPnP servers 43 via split screen device 40 perform notification messageMulticast is distributed, therefore the ACR of the second screen apparatus 50 can not receive notification message using 51.Therefore, in the step s 21,ACR can search for the ACR clients used using 51 search by the M defined by UPnP.
In step S31, device description is responded in response to the UPnP servers 43 of the split screen device 40 of M search.?In step S22, ACR is described using 51 reception devices.
By above-mentioned processing, the ACR of the second screen apparatus 50 applies 51 dresses that can obtain split screen device 40Set description.
[example of device description]
Here, the example of description device description.In order to describe device description, for example, using XML file.
Hereinafter, the first example of description device description.
First example of device description
It should be noted that the device specified in existing general UPnP described<SCPDURL>In element, description is directed toThe acquisition destination-address of the service describing for the ACR service describings that ACR clients 42 can use, and ACR is used using 51HTTP GET obtain service describing from destination-address is obtained.In addition, ACR will be moved using 51 using SOAP specified in UPnPThe message of work is sent to<controlURL>URL described in element, to call the action (in the present case, to send ACR inquiriesIt is responded with ACR is transmitted).
However, in the present embodiment, in order to which the step call action of quantity reduction can be used, as described above in UPnPDefined this device description is extended so that the WebAPI directly described in device description for calling the action.
In particular, as in the case of above-mentioned first example, for example,<SCPDURL>Element remains sky, inAny content is not described, is otherwise existed<SCPDURL>Scheduled virtual character string described in element, and directly exist<controlURL>It is used to start ACR clients described in element and asks to transmit the server of the WebAPI of ACR responses 120This URL of batter (, https exemplary in the case that first://192.168.0.4:55500/getACRResponse).
In addition, provide again does not have description in UPnP<WindowStyle>Element so that directly exist<WindowStyle>It is used for described in element to 45 notification frame perceptual property of window manager or changes frame perceptual propertyServer side scripting URL (, the https exemplary in the case that first of WebAPI://192.168.0.4:55500/windowStyle)。
It should be noted that for each of the ACR clients 42 of installation, description<service>Element.Therefore, at thisIn the case of embodiment,<serviceList>Described in three<service>Element.
Alternatively, it can provide what UPnP was not specified by again<WebAPIURL>Element so that such as following second is exemplaryIt is the same in situation, directly exist<WebAPIURL>The server side scripting URL of WebAPI described in element.
Second example of device description
Having obtained the ACR of the second screen apparatus 50 of the first example or the description of the second exemplary device can be with using 51So that executing action by calling getACRResponse, getACRResponse is described using HTTP GET in device<controlURL>Element or<WebAPIURL>The server side scripting url of WebAPI described in element.
In particular, if the ACR clients 42 of split screen device 40 start not yet, ACR visitors can be startedFamily end 42 so that ACR clients 42 send ACR inquiries 110, and transmit the ACR being sent back in response to ACR inquiries 110 and ringAnswer 120.It should be noted that (for example, with following in the state of described in the main body of the http response returned in response to WebAPIThe mode using JSON) execute ACR response 120 transmission.
HTTP/1.1200 OK
Content-Type:application/json；Charset=utf-8
" passing through the content of the ACR responses of JSON codings "
By the way that repeatedly poll, execution use the affairs of WebAPI while watching content.Then, in response to contentProgress passes through associated metadata 127 etc. of second screen apparatus 50 into the current converted ACR responses 120 of user's presentation.
It extends if as discussed above and modular station describes, then the multiwindow screen for being manufactured and being sold by different suppliersProscenia 40 and the second screen apparatus 50 can each other in incidence relation fast operatings.
It should be noted that it is the WebSocket formulated by IETF (internet engineering task group) that can use its standardizationEtc. replacing using WebAPI.Using WebSocket in the case of, local Web server 44 (or ACR clients 42 orSecond screen apparatus 50) in provide WebSocket servers so that ACR passes through using 51 and ACR clients 42WebSocket servers are sent out session (TCP connection) and establish request to establish session, and existed using above-described JSON etc.ACR responses 120 are transmitted in session.
In this case, if ACR clients 42 receive new ACR from ACR servers 24 responds 120, ACR responses120 are immediately passed ACR by the session of foundation applies 51.Therefore, it eliminates to required using WebAPIPoll needs, and independent of the ACR of poll period response 120 transmission immediately become possible.
In addition, ACR can be by calling windowStyle by local Web servers 44 from window manager using 5145 obtain the frame perceptual property for the window being arranged on the screen of split screen device 40, and windowStyle is to pass through HTTPGET is described in device<WindowStyle>The server side scripting url of WebAPI described in element.
In particular, the frame perceptual property described in the main body of the http response returned in response to WebAPI.For example, ifColor is arranged to frame perceptual property, then the border-color described in the main body of http response:#ffa000->It is specified orangeThe example of border color.
If should be noted that the frame perceptual property returned from split screen device 40-1 and from split screen deviceThe frame perceptual property that 40-2 is returned overlaps each other, then one in split screen device 40-1 and 40-2 can be asked to change framePerceptual property.
In particular, can be by right<WindowStyle>The HTTP POST requests of the URL of the WebAPI of element, willThe frame perceptual property frame perceptual property to be become is described in the main body of request.The frame perceptual property of following example instruction overlappingBecome (border- color:#ffa001) the case where.
POST/windowStyle HTTP/1.0
host:192.168.0.4:55500
Content-Length:20
border-color:#ffa001
[operations of ACR system using services 10]
Now, the operation of description ACR system using services 10.
It should be noted that the prerequisite as operations described below, it is assumed that in domestic network system 30, the second screen dressThe ACR for setting 50 has obtained device description using 51 from the UPnP servers 43 of split screen device 40.
In addition, it is assumed that in the ACR servers 24 of supply method, system 20, and can be watched on split screen device 40The corresponding ACR reference datas of each content 100 generated and be registered in ACR databases 63.
Figure 11 is the flow chart for the operation for showing ACR system using services 10.
In step S41, the content server 21 of supply method, system 20 begins through broadcasting network 11 and sends content.
On the other hand, in step S51, in domestic network system 30, split screen device 40 starts to receive and reproduceContent.Here, if the second screen apparatus of step S61 50 ACR using 51 call requests start ACR clients 42 andThe WebAPI of ACR responses is transmitted, then ACR clients 42 etc. are opened in response to WebAPI under the control of local Web server 44Begin to operate.
In particular, just being watched from content renderer 41 if the ACR clients 42 of split screen device 40 receiveContent supply, then in step S52 signature extraction part 71 signed data 114 is extracted from the content and by number of signatureIt is output to ACR inquiries according to 114 and generates part 72.In step S53, it includes signed data 114 that ACR inquiries, which generate part 72 and generate,ACR inquiries 110, and the ACR of generation inquiry 110 is sent to ACR servers 24 by communications portion 73.
In step S42, from the ACR servers 24 that ACR clients 42 receive ACR inquiries 110, ACR responds generating unitPoints 64 with reference to ACR databases 63, to identify and include in signed data 114 in the ACR inquiries 110 received is correspondingHold.In addition, in step S43, ACR responses generate part 64 and generate ACR responses 120 in response to the recognition result of the content, andAnd the ACR responses 120 of generation are sent to ACR clients 42 by communications portion 65.
In step S54, the ACR clients 42 that ACR responses 120 are received from ACR servers 24 ring the ACR received120 are answered to be supplied to local Web server 44.ACR responses 120 are transmitted to the second screen apparatus 50 by local Web server 44ACR applies 51.
In step S62, receive be passed to ACR using 51 ACR responses 120 ACR using 51 in, the second screenThe ACR of device 50 calls the WebAPI of the notice for claim frame perceptual property using 51.In response to this, in step S55, thisGround Web servers 44 notify the frame perceptual property managed by window manager 45 to apply 51 to ACR.
The ACR of the notice of receiving frame perceptual property notifies frame perceptual property to window manager 53 using 51.Window pipeReason device 53 confirm the frame perceptual property that receives whether with the frame perceptual property that is notified via other split screen devices 40Overlapping.If detecting overlapping, ACR asks to change frame perceptual property using 51 by HTTP POST requests.In stepS56, the request are sent to window manager 45 from local Web server 44.If window manager 45 changes frame according to requestPerceptual property, then the frame perceptual property after changing are transmitted to ACR from local Web server 44 as notice and apply 51.?Step S64, the ACR for receiving the frame perceptual property after changing notify the frame perceptual property after change to window using 51Mouth manager 53.The screen of hereafter associated application to be shown is arranged based on the frame perceptual property after change for window manager 53Frame perceptual property of image etc..
It should be noted that if be not overlapped in step S62 acknowledgement frames perceptual property, do not ask to change frame vision categoryProperty, and execute the processing of step S64.
Furthermore it is possible to the processing of step S62 to S64 is immediately performed after the processing of step S61, without waiting for stepThe processing (transmission of ACR responses) of S54.
The associated metadata that ACR includes using 51 ACR responses 120 is transmitted to using 51 analyses in step S65, ACR,And make the associated application that the acquisition of associated application manager 52 will follow the progress for the content watched and execute.
In step S66, associated application manager 52 is obtained and executed passes through step S44's by application server 23The supplied associated application of reason.It should be noted that if having obtained associated application, obtaining step can be omitted.In addition, such asFruit has been started up associated application, then can be omitted the starting step of associated application.In addition, associated metadata is presented to user sometimes127 information for including, without regard to the execution of associated application.
At this point, the frame of the screen picture of the associated application presented to user has by window manager 53 so that with more windowsThe consistent frame perceptual property of the frame perceptual property of the frame of the window of display corresponding content on mouthful screen apparatus 40.
Be repeatedly carried out when watching content the processing of above-mentioned step S41 to S44, the processing of step S51 to S56 andThe processing of step S61 to S66.However, it is possible to the execution of the processing in the processing and step S55 and S56 of step S61 to S66It is omitted in journey and repeats or reduce frequency.
Therefore, the second screen apparatus 50 can be according to the content just watched on split screen device 40 by userProgress operates associated application.
In addition, user can quickly grasp corresponds to more windows by the screen picture that the second screen apparatus 50 is presented to userWhich content in a plurality of content shown on mouth screen apparatus 40.
Incidentally, although supplier can be constituted by a series of server of above-mentioned processing is individually performed by hardwareThe content server 21 of system 20 to ACR servers 24 and domestic network system 30 split screen device 40 and the second screenDevice 50, but they can also be realized by executing the computer of software.The computer includes being integrated into specialized hardwareComputer, such as personal computer that can be by installing the universal class that various programs perform various functions.
Figure 12 is the exemplary block diagram for the hardware construction for showing above computer.
In computer 200, (arbitrary access is deposited by CPU (central processing unit) 201, ROM (read-only memory) 202 and RAMReservoir) it 203 is connected with each other by bus 204.
In addition, input/output interface 205 is connected to bus 204.Importation 206, output par, c 207, storage section208, communications portion 209 and driver 220 are connected to input/output interface 205.
Importation 206 is made of keyboard, mouse, microphone etc..Output par, c 207 is by structures such as display unit, loud speakersAt.Storage section 208 is made of hard disk, nonvolatile memory etc..Communications portion 209 is made of network interface etc..Driver220 driving removable mediums 211, such as, disk, CD, magneto-optic disk and semiconductor memory.
In the computer 200 that is constituted in a manner of this as described above, CPU 201 passes through 205 He of input/output interfaceThe program being stored in such as storage section 208 is loaded into RAM 203 by bus 204, and execute the program to execute onState a series of processing.
For example, the program to be executed by computer 200 (CPU 201) can be recorded to as the removable of encapsulation mediumOn medium 211 and it is arranged to removable medium 211.Furthermore it is possible to wide by such as LAN, internet or digital satelliteThe wired or wireless transmission medium broadcast provides program.
In computer 200, input/output can be passed through by the way that removable medium 211 to be loaded into driver 220Program is installed in storage section 208 by interface 205.Furthermore it is possible to pass through wired or wireless transmission medium by communications portion 209Program is received, and program is installed in storage section 208.Alternatively, program can be pre- installed appropriately ROM 202 or storageIn part 208.
It should be noted that the program to be executed by computer 200 can be as described in this specification successively chronologicallyThe program of processing is executed, or concurrently or if necessary carves the program for executing processing (such as, when calling processing).
It should be noted that the embodiment of present disclosure is not limited to above-mentioned specific embodiment, but can without departing fromVariously change in the case of the theme of present disclosure.
[list of numerals]
10 ACR system using services, 11 broadcasting networks, 12 bidirectional communication networks, 20 supply method, systems, 21 contentsServer, 22 meta data servers, 23 application servers, 24 ACR servers, 30 domestic network systems, 40 multiwindowsScreen apparatus, 41 content renderers, 42 ACR clients, 43 UPnP servers, 44 local Web servers, 50 second screensProscenia, 51 ACR applications, 52 associated application managers, 61 content obtaining parts, 62 signature extraction parts, 63 ACR numbersAccording to library, 64 ACR responses generate part, and 65 communications portion, 71 signature extraction parts, 72 ACR, which are inquired, generates part, and 73 is logicalBelieving that part, 74 ACR respond holding part, 100 ACR reference datas, 110 ACR are inquired, 120 ACR responses, 200 computers,201 CPU