Background technology
In the past few years, digital video monitoring turns into a critically important application.The technology being directed to, includingThe essential Video coding of video surveillance applications and browser technology, obtain increasing concern.In video monitoring sideFace, most of videos are all captured by fixing camera；Even for on-fixed camera, most of video segmentsIt is still to be recorded from fixed viewpoint and fixed position in the set time.Therefore, video technique by using this characteristic be regardingFrequency monitoring provides a more efficient scheme.
In monitor video field of storage, many scholars are devoted to improve storage efficiency and increase the adaptability of code stream.It is rightSimilarly goed deep into the monitor video scalable coding technology towards different channels, different terminals, different viewing demandResearch.However, scalable coding technology solve only the flexible time domain based on frame to tradition, spatial domain is scalable and quality can stretchThe problem of contracting is browsed.And how more easily to retrieve and browse monitor video this problem and do not solve still to a certain extent.
Therefore, video frequency searching and video browsing techniques are progressively valued by the people, and have been applied in video monitoring system.Video summarization technique（A.Rav-Acha,Y.Pritch,and S.Peleg.Making a long video short:Dynamic video synopsis.In CVPR’06,volume1,pages435–441,2006.）It is used as the total of longer videoKnot enables people to access multitude of video content in the shorter time.Therefore, this is a very effective monitored video browseMethod.Unlike traditional video summarization technique, the dynamic video of video frequency abstract is concentrated, can be original original by showing simultaneouslyThe object motion occurred when different in video is come while maximum useful content is retained that original video boil down to one is short-sightedFrequently.Further, in order to meet video processing in real time and the demand of fast browsing, plum green grass or young crops et al. proposes the online of video concentrationHandle framework（Feng S,Lei Z,Yi D,et al.Online content-aware video condensation[C]//Computer Vision and Pattern Recognition(CVPR),2012IEEE Conference on.IEEE,2012:2082-2087.）.And a telescopic video concentration technique based on details is suggested and is used for ladderingly prominent regardFrequency content and raising storage efficiency（Wang S,Yang J,Zhao Y,et al.A surveillance video analysisand storage scheme for scalable synopsis browsing[C]//Computer VisionWorkshops(ICCV Workshops),2011IEEE International Conference on.IEEE,2011:1947-1954.）.
In addition, in the patent of invention of Application No. 201110346398.0, describing a kind of support video scalability clearThe video coding-decoding method and device look at, it is related to object flag position, for the mark to original video encoding and decoding video data,Moreover, video concentration technique has been further related to, the fast browsing for monitor video.The enough effective expressions of described object flag potential energyMapping relations of the area information and Moving Objects of Moving Objects between original video and concentration video.In simple terms, it is rightAs flag bit includes subject area flag bit and object map flag bit, their specifying information is：1) subject area flag bit,The Pixel-level mask that video analysis obtains Moving Objects is first passed around, and a boundary rectangle frame is generated based on mask, obtains outerRectangle frame information is connect, then merges object mask information with the macro block division information in the code stream of monitored video compression domain, is utilizedBlock division information in code stream carries out the mark of sub-block level accuracy to Moving Objects；2) object map flag bit, concentrates videoThe Moving Objects that different time sections in original video occur can be included in same frame, therefore object map flag bit is designedTo record mapping relations of the Moving Objects between original video and concentration video, that is, it marked the motion pair in original videoAs which frame in concentration video is shown.
Relative to video browsing techniques, merging the research of the video browsing techniques of video retrieval technology will lack.But, work as handleWhen video retrieval technology is applied to video monitoring system, it can give people to provide more targeted monitored video browse effect,Improve the service efficiency of monitor video.
The content of the invention
It is an object of the invention to provide a kind of video frequency searching based on object flag position and the method and apparatus that browse,Do not influence on the basis of monitor video code efficiency, realize the efficient retrieval and fast browsing of monitor video.
The present invention is realized using following technical scheme.The present invention provides a kind of video inspection based on object flag positionRope and the method browsed, comprise the following steps：S1, it is dense based on the original video progress stored in memory module that video concentrates moduleContracting generation concentration video A, and it is stored in memory module；S2, the video concentration module generation of object flag position module in step slDuring concentrating video A, based on the original video in memory module, subject area flag bit and object map mark are obtainedPosition, and described zone bit information is stored in memory module；S3, retrieval module is retrieved satisfactory based on object flag positionVideo segment；S4, video concentrates module and concentration generation concentration video B is carried out to the video segment retrieved, and is stored in storageModule；And S5, browsing module carries out video to the concentration video B generated in the concentration video A or step S4 that are generated in step S1Concentration is browsed.
The present invention can be also applied to the following technical measures to achieve further.
Preferably, the foregoing video frequency searching based on object flag position and the method browsed, wherein in step s 2, obtainingSubject area flag bit be Moving Objects in original video area information, and it is original video to obtain object map flag bitThe correspondence mappings information of Moving Objects between concentration video.
Preferably, the foregoing video frequency searching based on object flag position and the method browsed, wherein in step s 2, working as fortuneWhen dynamic object is mapped to concentration video A, priority of the Moving Objects in video frequency searching can be high, and in step s3, inspectionThe rope module Moving Objects high to priority are retrieved, so as to while retrieval effectiveness is ensured, further reduce and treatThe data volume of retrieval.
Preferably, the foregoing video frequency searching based on object flag position and the method browsed, wherein in step s 2, may be used alsoTogether with above-mentioned subject area flag bit, object map flag bit and original video to be worked out, original video is set to turn into baseMonitor video in object flag position.
Preferably, the foregoing video frequency searching based on object flag position and the method browsed, wherein in step s3, retrievalModule can select the mode specifically retrieved, and analyze the specifying information of target to be retrieved and the moving target in memory module, and rightThan the matching degree of the information of the moving target in target to be retrieved and memory module, and then retrieve satisfactory piece of videoSection.
, can be with wherein in step s3 preferably, foregoing video frequency searching and the method that browses based on object flag positionThe mode retrieved includes sample retrieval, color retrieval, Shape-memory behavior and trajectory retrieval.
Preferably, the foregoing video frequency searching based on object flag position and the method browsed, wherein in step s 5, it is describedBrowsing module can also carry out it is scalable concentration browse.
The present invention is also realized using following technical scheme.The present invention also provides a kind of video inspection based on object flag positionRope and the device browsed, including：Object flag position module, video concentration module, memory module, retrieval module and browsing module；Object flag position module is connected to video concentration module and memory module, the fortune of the original video for concentrating video to generationThe scheme moved subject area and its be mapped to concentration video is marked, to obtain subject area flag bit and object map markPosition information, and it is deposited into memory module；Video concentration module is connected to memory module, for original video or having retrievedThe video segment gone out carries out video concentration, and the concentration video of generation is stored in into memory module；The retrieval module is connected to storageModule, can carry out the retrieval of different modes, for retrieving satisfactory video segment；The browsing module is connected to storageModule, can carry out retrieval concentration and browse.
The present invention can be also applied to the following technical measures to achieve further.
Preferably, the foregoing video frequency searching based on object flag position and the device browsed, wherein object flag position mouldBlock includes subject area mark bit submodule and object map mark bit submodule, and the subject area mark bit submodule is used to obtainThe area information of Moving Objects in original video is taken, the object map mark bit submodule is used to obtain original video and concentration is regardedThe correspondence mappings information of Moving Objects between frequency.
Preferably, the foregoing video frequency searching based on object flag position and the device browsed, wherein object flag position mouldBlock also includes video compiling module, for above-mentioned subject area flag bit, object map flag bit and original video to be worked outTogether, original video is made to turn into the monitor video based on object flag position,
Preferably, the foregoing video frequency searching based on object flag position and the device browsed, wherein memory module are mainFor storing original video, the monitor video based on subject area flag bit and object map flag bit, retrieval threshold and having examinedThe video segment that rope goes out.
Preferably, the foregoing video frequency searching based on object flag position and the device browsed, are wherein deposited in the memory moduleThe retrieval threshold of storage can both be set in advance, can be inputted in real time again, while the difference according to retrieval mode, each retrieval thresholdIt is also different.
Preferably, the foregoing video frequency searching based on object flag position and the device browsed, wherein retrieval module includeCharacteristics analysis module, comparative analysis module and input module；The input module is connected to characteristics analysis module, this feature analysis mouldBlock is connected to comparative analysis module and memory module again, and comparative analysis module is connected to memory module；The input module is usedThe mode specifically retrieved in selection, this feature analysis module analyzes target to be retrieved and storage according to the retrieval mode selectedMotion in the specifying information of moving target in module, the comparative analysis module comparative analysis target to be retrieved and memory moduleThe similarity or matching degree of the information of target.
Preferably, the mode bag of foregoing video frequency searching and the device, the wherein retrieval that browse based on object flag positionInclude sample retrieval, color retrieval, Shape-memory behavior and trajectory retrieval.
By above-mentioned technical proposal, video frequency searching of the present invention based on object flag position and the device and method that browses are at leastWith following advantages and beneficial effect：By the present invention, prison can be realized on the basis of monitor video code efficiency is not influenceedControl the efficient retrieval and fast browsing of video.
Described above is only the general introduction of technical solution of the present invention, in order to better understand the technological means of the present invention,And can be practiced according to the content of specification, and in order to allow the above and other objects, features and advantages of the present invention canBecome apparent, below especially exemplified by preferred embodiment, and coordinate accompanying drawing, describe in detail as follows.
Embodiment
Further to illustrate the present invention to reach the technological means and effect that predetermined goal of the invention is taken, below in conjunction withAccompanying drawing and preferred embodiment, to according to a kind of video frequency searching and the device that browses based on object flag position proposed by the present invention andEmbodiment, structure, feature and its effect of method, are described in detail as after.
As shown in fig.2, the block diagram of the video frequency searching and the device browsed based on object flag position for the present invention.ShouldDevice includes object flag position module 1, video concentration module 2, memory module 3, retrieval module 4 and browsing module 5.
Object flag position module 1 is connected to video concentration module 2 and memory module 3, for concentrating video to generationThe Moving Objects region of original video and its it is mapped to the scheme of concentration video and is marked, including subject area flag bit submoduleBlock 11 and object map mark bit submodule 12.The subject area mark bit submodule 11 is used to obtain motion pair in original videoThe area information of elephant, the area information has marked region of the Moving Objects in original video, i.e. subject area markPosition, and memory module 3 is deposited into, these combine original video, can obtain texture, color, shape and the fortune of Moving ObjectsDynamic rail mark.The correspondence that the object map mark bit submodule 12 is used to obtain Moving Objects between original video and concentration video is reflectedInformation, i.e. object map flag bit are penetrated, and is deposited into memory module 3.In addition, object flag position module 1 also includes videoCompiling module 13, for above-mentioned subject area flag bit together with object map flag bit and original video establishment, to be madeOriginal video turns into the monitor video based on object flag position, and the monitor video deposit based on object flag position is stored into mouldBlock 3, is browsed beneficial to follow-up quick-searching and concentration.
In the present embodiment, if the Moving Objects in original video are not mapped onto concentration video A, it can be understood asThe significance level of the Moving Objects is relatively low, and so its priority in video frequency searching is relatively low, conversely, when Moving Objects are reflectedWhen penetrating, priority of the Moving Objects in video frequency searching is also higher.It is worth noting that, the present invention pair high to priorityAs region is retrieved, so as to while retrieval effectiveness is ensured, further reduce data volume to be retrieved.
Video concentration module 2 is connected to memory module 3, can be not only used for carrying out original monitor video video concentration, alsoConcentrated available for the video segment to having retrieved.Video concentration, which is one, is used for the brand-new of monitor video fast browsingMethod of abstracting, it is by putting the object extracted from different frame together so as to effectively reduce space-time when browsing original videoDomain redundancy.Specific video concentration method may refer to documents below：A.Rav-Acha,Y.Pritch,andS.Peleg.Making a long video short:Dynamic video synopsis.In CVPR’06,volume1,pages435–441,2006.
The memory module 3 is mainly used in storing original video, the monitor video based on object flag position（Object flag positionRefer to subject area flag bit and object map flag bit）, retrieval threshold and the video segment retrieved, the wherein retrievalThreshold value can both be set in advance, can in real time be inputted by the input module of retrieval module again.Specifically, according to memory module 3In subject area flag bit, can obtain the Moving Objects area information in original video, and pair in memory module 3As mapping flag bits, the priority of subject area and generation concentration video A mapping scheme can be obtained.
As shown in fig.3, the block diagram of the retrieval module for the present invention.The retrieval module 4 is connected to memory module 3, bagInclude characteristics analysis module 41, comparative analysis module 42 and input module 43；The input module 43 is connected to characteristics analysis module 41,This feature analysis module 41 is connected to comparative analysis module 42 again, and comparative analysis module 42 is connected to memory module 3.The inputModule 43 is used for the mode for selecting specifically to retrieve.This feature analysis module 41 is analyzed to be retrieved according to the retrieval mode selectedThe specifying information of target and the moving target in memory module 3.The comparative analysis module 42 is used to contrast target to be retrieved and depositedSimilarity or matching degree of the information of moving target in storage module 3 etc., if reach threshold value., can be by when meeting the requirementsThe video segment deposit memory module 3 retrieved.
In the present embodiment, the retrieval mode related generally to is sample retrieval, color retrieval, Shape-memory behavior and track inspectionRope.Above-mentioned four kinds of retrieval modes will be described in detail below：
Sample is retrieved：By input module 43, user directly inputs the sample picture of a target to be retrieved；Subsequent featureAnalysis module 41 receives the sample picture, and extracts the textural characteristics of sample picture（Specific extracting method, referring to textOffer：P.Wu,B.S.Manjunath,S.D.Newsam,and H.D.Shin,“A texture descriptor for imageretrieval and browsing,”in Computer Vision and Pattern Recognition Workshop,Fort Collins,CO,Jun.1999,pp.3–7.）, while can also extract each moving target in memory module 3Textural characteristics, and it is sent to comparative analysis module 42；The textural characteristics of the comparative analysis sample picture of comparative analysis module 42 andThe textural characteristics of each moving target in memory module 3, set in advance when exceeding with the matching degree of some moving targetFixed threshold value（More than or equal to 80%）, then it represents that meet the requirements, and the moving target is stored in memory module 3.
Refering to shown in Fig. 4 A, by input module 43, the figure of a car is directly inputted after user's selection sample retrievalPiece, characteristics analysis module 41 receives the car picture, and extracts the textural characteristics of car picture, subsequent comparative analysisThe textural characteristics of the comparative analysis car picture of module 42 and the textural characteristics of each moving target in memory module 3, whenMatching degree with some moving target is more than 80%, then it represents that meet the requirements, and the Moving Objects are stored in into memory module 3.Figure4A is the subjective effect schematic diagram of the car retrieved using sample.
Color is retrieved：Color is Moving Objects the most intuitively one of feature in monitor video, in the present invention it is possible to logicalCross simple color histogram（Referring specifically to document：B.S.Manjunath,J.-R.Ohm,V.V.Vasudevan,andA.Yamada,“Color and Texture Descriptors,”IEEE Trans.Circuits and Systems forVideo Technology,11(6):703–715,2001.）The color for matching to carry out Moving Objects is retrieved.In the present invention,The monitor video decoded has been converted to RGB color.By input module 43, selection carries out color retrieval, subsequent featureAnalysis module 41 analyzes the color of target to be retrieved, and the comparative analysis of comparative analysis module 42 target to be retrieved and memory module 3In each moving target RGB color matching degree, when the matching degree average value between each Color Channel is more thanSome value（Threshold value is 70%）When, it is defined as being mutually matched, otherwise represents to mismatch.Finally, the Moving Objects deposit of matching is selectedMemory module 3.
Refering to shown in Fig. 4 B, by input module 43, choosing is inputted in the palette of system after user's selection color retrievalFixed color, characteristics analysis module 41 is received after selected color, by the comparative analysis of comparative analysis module 42 input color andThe matching degree of three color spaces of each moving target in memory module 3, the matching degree between each Color ChannelAverage value is more than 70%, is defined as being mutually matched and is stored in memory module 3.Fig. 4 A are the cars retrieved using colorSubjective effect schematic diagram.
Shape-memory behavior：Shape-memory behavior is mainly is retrieved or is directly inputted mesh by the profile diagram of Freehandhand-drawing destination objectThe profile diagram of mark object is retrieved.By input module 43, selection carries out Shape-memory behavior, and subsequent characteristics analysis module 41 is analyzedThe shape of target to be retrieved, and each motion in the comparative analysis of comparative analysis module 42 target to be retrieved and memory module 3The matching degree of the shape of target.Finally, the Moving Objects deposit memory module 3 of matching is selected.
Specifically, in the present invention, the shape of the moving target of each marked area can use 7 degree of freedom Characteristic Vectors scaleShow.（Specific generation method referring to：M.Hu,“Visual pattern recognition by moment invariants,”IRE Trans.Inform.Theory,vol.IT-8,no.2,pp.179–182,Feb.1962.）
If moving target A shape representation is：And moving target B shape representation is：Similarity between moving target A and B shape can be calculated by following formula：
When the similarity between moving target A and B shape exceedes threshold value 60%, moving target A and B phase is considered asMatch somebody with somebody.
Refering to shown in Fig. 4 C, by input module 43, user's selection carries out Shape-memory behavior and Freehandhand-drawing and goes out carProfile diagram, characteristics analysis module 41 analyzes this profile diagram and simultaneously extracts information, the subsequent comparative analysis of comparative analysis module 42 thisThe matching degree of the shape of profile diagram and each moving target in memory module 3, when similarity is more than 60%, then it is assumed that matchingSuccess, is stored in memory module 3.Fig. 4 C are the subjective effect schematic diagrames of the car retrieved using Shape-memory behavior.
In fact, can easily search out the motion pair with Representative character based on Moving Objects shape-based retrievalAs such as pedestrian, the type games object of vehicle two has obvious shape difference.But, the shape of Moving Objects directly can be with motionThe direction of motion of object is related to the angle of monitoring camera, and same non-rigid motion object is in video it is possible that veryObvious change in shape, these factors also cause Shape-memory behavior to have some limitations.
Trajectory retrieval：In addition to above-mentioned static nature can be retrieved, the present invention can also carry out the rail of Moving ObjectsMark is retrieved, and trajectory retrieval is different with the feature matching method of other modes, because the track of Freehandhand-drawing has time domain specification, so railThe matching process of mark need not frame by frame be matched to Moving Objects, but the set to overall point carries out retrieval matching.（It can joinSee document：JunWei Hsieh,Shang-Li Yu and Yung-Sheng Chen,Motion-based videoretrieval by trajectory matching,IEEE Transactions on Circuits and Systemsfor Video Technology,Vol.16,No.3,396-409,2006.）
Concrete implementation step is as follows：
If the track of Moving Objects is expressed as：
Wherein mkThe track points of k-th of Moving Objects are represented, N is Moving Objects sum to be retrieved.User's Freehandhand-drawingTrack is expressed as：
Wherein mRRepresent the coordinate points of retrieval track.
General, the coordinate points m of the retrieval track of inputRWith the track points m of Moving ObjectskBetween have substantially it is poorNot.Therefore, if calculating the distance between two track each points, then just very possible erroneous judgement, miss what can be matched originallyMoving Objects.Thus, our tracks to target trajectory and target to be retrieved are normalized：
If mk＞ mR, calculate proportionality coefficient Ra=mk/mR, then by TkCoefficients R a abandon obtaining T ' in proportionk, thenCarry out Similarity Measure between points.If likewise, mk＜ mR, it would however also be possible to employ similar method, carry out track pointsThe Similarity Measure between track is carried out after normalization again.
When the similarity between track exceedes threshold value 50%, it is considered as matching with searched targets, and be stored in memory module.
Refering to shown in Fig. 4 D, by input module 43, user's selection carries out trajectory retrieval and Freehandhand-drawing goes out motion to be retrievedObject trajectory, characteristics analysis module 41 analyze the Moving Objects track, the subsequent comparative analysis of comparative analysis module 42 track andThe trace information of each moving target in memory module 3, the moving target that selection matching degree is more than 50% is used as successful matchAs a result, and it is stored in memory module 3.Fig. 4 D are the subjective effect schematic diagrames of the car retrieved using trajectory retrieval.
The browsing module 5 is connected to memory module 4, and the retrieval concentration that can carry out retrieval result is browsed.Specifically, originallyThe browse mode being related in invention is browsed for retrieval concentration, and it is required mainly on the basis of retrieval for meeting retrievalVideo segment, module 2, generation concentration video are concentrated by video, and are browsed, so that new retrieval concentration video is moreWith specific aim.Browsed in fact, the browsing module 5 can also carry out scalable concentration., should in order to realize that scalable concentration is browsedBrowsing module 5 realizes a kind of scalable method for reconstructing based on object map flag bit, and this method can adjust Moving ObjectsMapping relations, and reconstruct a variety of concentration videos of different length, different densities and different fidelity.And user can rootAccording to viewing demand, set scalable series to reconstruct the concentration video of oneself satisfaction.It for details, reference can be made to document：Wang,S.,Yang,J.,and Stan Z.Li,A Surveillance Video Analysis and Storage Scheme forScalable Synopsis Browsing.ICCV Workshop on Visual Surveillance,November,2011.
The video frequency searching based on object flag position to the present invention is described in detail with the method browsed below, itsMainly include the following steps that（Refering to shown in Fig. 1）：
S1, video concentrates module 2 and carries out video analysis based on the original video stored in memory module 3, obtains motion pairThe area information of elephant, and then video concentration is carried out, obtain the mapping scheme of Moving Objects in original video and concentration video, generationVideo A is concentrated, and will concentration video A deposit memory modules 3；
During S2, the video concentration module generation concentration video A of object flag position module 1 in step sl, it is based onThe area information and mapping scheme of the Moving Objects of acquisition, generation subject area flag bit and object map flag bit, and by instituteThe zone bit information deposit memory module 3 stated；
Further using the video compiling module 13 in the module 1 of object flag position, by the subject area mark in step S2Together with will position, object map flag bit are worked out with original video, turn into original video and marked fortune by object flag positionThe monitor video of dynamic subject area and Moving Objects priority, and the monitor video deposit based on object flag position is stored into mouldBlock 3, so that follow-up quick-searching is browsed with retrieval concentration；
S3, by the input module 43 of retrieval module 4, is selected after the mode specifically retrieved, by 41 points of characteristics analysis moduleThe specifying information of target to be retrieved and the moving target in memory module 3 is analysed, and by comparative analysis module 42, is contrasted to be retrievedSimilarity or matching degree of target and the information of the moving target in memory module 3 etc., and the video segment retrieved is depositedEnter memory module 3；Wherein, the retrieval of different modes, such as sample retrieval, color retrieval, Shape-memory behavior can be carried out hereinAnd trajectory retrieval；
S4, the video segment that 2 pairs of module of video concentration has been retrieved, which carries out concentration, turns into concentration video B, and is stored in storageModule 3；
S5, browsing module 5 is carried out to the concentration video B generated in the concentration video A or step S4 that are generated in step S1It has been shown that, and browse mode browses or retrieved concentration for scalable concentration and browses.
By video frequency searching of the present invention based on object flag position and the device and method that browses, prison can not influenceedControl on the basis of video coding efficiency, realize the efficient retrieval and fast browsing of monitor video.
The above described is only a preferred embodiment of the present invention, any formal limitation not is made to the present invention, thoughSo the present invention is disclosed above with preferred embodiment, but is not limited to the present invention, any to be familiar with this professional technology peopleMember, without departing from the scope of the present invention, when the technology contents using the disclosure above make a little change or modificationFor the equivalent embodiment of equivalent variations, as long as being the content without departing from technical solution of the present invention, the technical spirit according to the present inventionAny simple modification, equivalent variations and the modification made to above example, in the range of still falling within technical solution of the present invention.