CN112347272A - A method and device for streaming matching based on dynamic features of audio and video - Google Patents
A method and device for streaming matching based on dynamic features of audio and videoDownload PDFInfo
- Publication number
- CN112347272A CN112347272ACN202010987148.4ACN202010987148ACN112347272ACN 112347272 ACN112347272 ACN 112347272ACN 202010987148 ACN202010987148 ACN 202010987148ACN 112347272 ACN112347272 ACN 112347272A
- Authority
- CN
- China
- Prior art keywords
- fingerprint
- interval
- hash table
- data
- length
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/41—Indexing; Data structures therefor; Storage structures
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/432—Query formulation
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/43—Querying
- G06F16/432—Query formulation
- G06F16/433—Query formulation using audio data
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
- G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/483—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- General Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Library & Information Science (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Information Transfer Between Computers (AREA)
Abstract
Description
Technical Field
The invention belongs to the technical field of high-speed network flow deep analysis, audio and video identification and dynamic fingerprint matching, and particularly relates to a streaming matching method and device based on audio and video dynamic characteristics.
Background
Audio-video recognition technology has a wide demand in the information age. Some enterprises or organizations with higher security requirements have stronger monitoring requirements on audio and video contents transmitted by the internet. And monitoring and auditing the audio and video flow entering and exiting the Internet to find out bad contents in the audio and video flow. For example, some entertainment audio/video website providers need to review the audio/video contents uploaded by users to find out the harmful audio/video programs such as pornography, reaction, violence, etc. Meanwhile, enterprises or organizations need to attack pirated audios and videos and protect the rights and interests of the pirated audios and videos by identifying the audios and the videos. Through the detection and audit of the audio and video flow, whether audio and video contents infringing the copyright exist is found, and the income of a copyright side is protected or the legal risk caused by piracy is avoided. In addition, enterprises or organizations have a need to detect audio and video similarity so as to find similar audio and video, remove repeated audio and video, and improve network utilization efficiency or audio and video analysis capability. In order to identify the audio and video transmitted in the network in time, fingerprint matching needs to be performed on each data packet in the audio and video transmission process in real time.
The existing audio-video feature matching is carried out aiming at the complete audio-video file. Patent CN107633078A proposes an audio frequency feature extraction and detection method, which extracts the energy maximum point of the whole audio frequency as a fingerprint, and then compares the fingerprint with the fingerprint features of other audio frequencies in the database to detect whether the audio frequencies are consistent. The patent CN111368143A proposes inputting the whole video segment into multiple 3D convolutional layers to obtain a first feature vector, and then comparing the first feature vector with the data in the video feature library to detect the similarity of the video. The disadvantages of the prior art are as follows:
1. randomness of network traffic. The monitoring system acquires audio and video flow in a bypass mode, and due to the reasons of flow mirroring, light splitting, transmission and the like, the network flow captured by the system has the phenomena of packet loss, disorder, incompleteness and the like, so that the calculation summary by acquiring the fixed segment is not feasible.
2. Because the existing audio and video identification method extracts fingerprints through the whole audio and video file or segment and then compares the fingerprints with data in a fingerprint database, matching identification is slow, and audio and video cannot be identified in time in high-speed flow.
3. The fingerprint characteristics of the audio-video may change according to the type of the audio-video to be recognized, because the type of the audio-video to be recognized by the user may change. In the current technical scheme, fingerprints which can distinguish audios and videos are extracted and compared. The addition and deletion of fingerprint features cannot be satisfied in order to identify videos of specific types and contents.
Disclosure of Invention
The invention provides a streaming audio and video dynamic fingerprint matching method in order to meet the requirements of examining audio and video contents containing violence, reaction and pornography, limiting the spread of bad videos, protecting original songs and video copyrights and reducing similar audios and videos in network flow. The method is based on the interval index tree and the dynamic hash table, solves the problem of quickly identifying the audio and video in high-speed flow, and meets the requirement of quickly matching the audio and video of enterprises or organizations. The method has the functions of detecting bad content, attacking pirated audios and videos, removing repeated audio and video streams and the like.
The technical scheme adopted by the invention is as follows:
a streaming matching method based on audio and video dynamic characteristics comprises the following steps:
establishing an interval index tree for all fingerprint intervals by taking the offset position as a standard, and storing fingerprint information of the audio and video by using the interval index tree;
and for the data block of the audio and video content to be matched, searching the interval index tree to find out the matched fingerprint and outputting a fingerprint matching result, thereby realizing audio and video identification.
Further, the interval index tree allocates fingerprints at different offset positions to different intervals, and places fingerprints at the same offset position in the same interval; the interval index tree comprises a plurality of interval nodes, and each interval node comprises an interval with the length of 1 and interval attached information; the interval auxiliary information comprises the length of the longest fingerprint and a first-level hash table; the first-stage hash table is a length hash table, the key of the first-stage hash table is the fingerprint length, and the value of the first-stage hash table is a second-stage hash table; the second-level hash table is a fingerprint hash table, the key of the second-level hash table is the binary content of the fingerprint, and the value is the configuration information of the fingerprint.
Further, adding a fingerprint to the span index tree using the following steps:
step 1021: searching the interval index tree according to the offset position of the fingerprint, if no interval node with the same offset position exists, establishing an interval node, establishing a two-stage hash table, and adding the current fingerprint information into the hash table; if there is an interval node with the same offset position, go to step 1022;
step 1022: searching two-stage hash tables to inquire whether the current node has fingerprints with the same length; if not, go to step 1024; if there are fingerprints with the same length and different binary contents, go to step 1023; if the fingerprint with the same length as the binary content of the fingerprint exists, the fingerprint is forbidden to be added, and the fingerprint adding is finished;
step 1023: inserting data into the fingerprint hash table, adding fingerprint binary content and fingerprint configuration information into the fingerprint hash table, and finishing fingerprint addition;
step 1024: creating a fingerprint hash table, adding the binary content of the fingerprint and the fingerprint configuration information into the fingerprint hash table, and simultaneously inserting the fingerprint length and the handle of the fingerprint hash table just created into the length hash table; and simultaneously updating the length of the longest fingerprint in the interval and finishing fingerprint addition.
Further, the specified fingerprint in the interval index tree is deleted by adopting the following steps:
step 1031: searching interval nodes with the same offset position, if not, returning to the invalid deletion and finishing the deletion, wherein the fingerprint which needs to be deleted does not exist; if yes, go to step 1032;
step 1032: searching a length hash table in a secondary hash table of the current node, if no fingerprint hash table handle with the corresponding length exists, indicating that no fingerprint with the length exists, deleting ineffectively, and finishing deleting; if so, go to step 1033;
step 1033: retrieving the fingerprint hash table by using the binary content of the fingerprint, if no corresponding fingerprint configuration information exists, then no corresponding fingerprint exists, deleting invalidity, and finishing deleting; if so, go to step 1034;
step 1034: deleting the corresponding fingerprint binary fingerprint and fingerprint configuration information in the fingerprint hash table, and checking whether the fingerprint hash table is empty; if not, finishing the deletion; if empty, go to step 1035;
step 1035: deleting the corresponding fingerprint length and the fingerprint hash table handle in the length hash table, updating the length of the longest fingerprint, and checking whether the length hash table is empty; if not, finishing deleting; if the node is empty, the node indicates that no fingerprint exists in the interval node, the node is deleted, and the deletion is finished.
Further, for the data block of the audio and video content to be matched, finding out the matched fingerprint through the search interval index tree and outputting a fingerprint matching result, including:
step 2001: streaming input audio and video content, and ending the matching process if the input is ended; otherwise, if the last matching residual cache region is not empty, merging the residual cache region with the current input data to serve as a data block to be matched, wherein the offset position of the data block is the offset position of the cache region; if the last matching residual cache area is empty, directly taking the current input as a data block to be matched;
step 2002: searching an interval index tree, and checking whether an interval with the length of 1 in interval nodes is included in the input data; if not, discarding the data; if yes, go to step 2003;
step 2003: judging whether the longest fingerprint in the interval nodes is contained in the data or not according to the length of the longest fingerprint of the contained interval nodes, namely the input data completely contains the fingerprint interval; if not, storing the data in the matching residual cache region, and entering step 2001 to wait for data input; if so, go to step 2004;
step 2004: traversing keys of the length hash table, finding fingerprint hash tables of all fingerprints in the interval nodes through the length hash table, and taking a section of data with the same offset position and length as the fingerprints in the data as the keys of the fingerprint hash tables to obtain a corresponding value; if the value does not exist, the data is not matched with the fingerprint, and the matching is finished; if value exists, the data is successfully matched with the fingerprint, and the success state and the fingerprint configuration information are output.
Further, in step 2001, the overlap between the input data and the fingerprint section in data merging includes 4 cases:case 1 is that the input data contains a fingerprint interval completely; case 2 is where the input data partially coincides with the fingerprint interval, and there is a portion of the fingerprint interval before the initial offset position of the input data; case 3 is that the fingerprint interval contains input data completely; case 4 is where the input data partially coincides with the fingerprint section and there is a portion of the fingerprint section after the offset position at the end of the input data.
Further, the data is merged intocase 1 in the following two ways:
1) case 4+ n case 3+ case 2: the method comprises the steps of gradually merging data when sequentially input data are respectively in a case 4, a case 3 and a case 2, the relation between the merged data and fingerprints is acase 1, and n represents a plurality of data;
2) case 4+ case 2: the data is merged when the data sequentially input is case 4 and case 2, respectively, and the relationship between the merged data and the fingerprint iscase 1.
A adopt the above-mentioned method a kind of stream-type matching device based on dynamic characteristic of the audio frequency and video, it includes:
the fingerprint management module is used for establishing an interval index tree for all fingerprint intervals by taking the offset position as a standard and storing the fingerprint information of the audio and video by using the interval index tree;
and the fingerprint identification module is used for searching the interval index tree for the matched fingerprint of the data block of the audio and video content to be matched and outputting a fingerprint matching result, so that the audio and video identification is realized.
The key points of the invention are as follows:
1. and detecting audio and video data messages. And the quick judgment of whether the data of each arbitrary offset position has a matched fingerprint or not is realized through the interval index tree and the two-stage hash table, and the matched state is output. The problems of low audio and video matching speed, random arrival of data packets and uncertain length are solved, real-time matching is achieved, and the detection speed is improved.
2. Dynamic fingerprint features. By means of establishing the interval index tree, dynamic management of fingerprint features is achieved, and users can dynamically add or delete the fingerprint features according to needs. The problem of fixed unchangeable finger print characteristic among the current technical scheme, can't change along with user's demand is solved.
The invention has the following beneficial effects:
1. by detecting each data message, the requirement for fixing the length of the audio/video clip is cancelled, and the application range is widened. Meanwhile, the feature extraction of the whole audio and video is avoided, and the detection speed is improved.
2. By using the two-stage hash table, whether the audio and video data are matched with the fingerprint can be rapidly detected, the fingerprint matching efficiency is greatly improved, and the method can adapt to the environment of high-speed large-flow network data.
3. And dynamic management of fingerprint features is realized through the interval index tree. The fingerprint characteristics can be changed according to the needs of users, and the requirement that the fingerprint characteristics are likely to change is met.
Drawings
FIG. 1 is a flowchart of the operation of a fingerprint management module.
FIG. 2 is a flowchart of the operation of the fingerprint identification module.
Fig. 3 is a schematic diagram of the overlapping of the input data and the fingerprint interval when data are merged.
FIG. 4 is a diagram illustrating a structure of interval nodes in an interval index tree.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, the present invention shall be described in further detail with reference to the following detailed description and accompanying drawings.
As shown in fig. 1 and 2, the streaming audio/video dynamic fingerprint detection framework can be mainly divided into two modules: the fingerprint management module (figure 1) establishes an interval index tree for all fingerprint intervals by taking an offset position as a standard, and then realizes storage and quick retrieval of all fingerprint information. And the system also comprises two sub-modules which are used for respectively processing the addition and the deletion of the fingerprint information and realizing the dynamic management of the fingerprint. And the fingerprint identification module (figure 2) is used for merging the input data into a proper size, searching the interval index tree, finding out the matched fingerprint and outputting a fingerprint matching result, so that the audio and video identification is realized.
The detailed steps of each sub-module are explained below.
1. Fingerprint management module
(1) Interval index tree construction
Step 1011: the data structure of the interval index tree nodes is determined as shown in fig. 4. The method comprises the steps of taking a section with the length of 1 (namely a section formed by a certain offset position in the audio and video) as a part of a node, distributing fingerprints with different offset positions to different sections, and placing the fingerprints with the same offset position in the same section. In order to distinguish fingerprints with the same offset position and quickly find fingerprint configuration information according to fingerprint content, a two-stage hash table is established. The first level of the two-level hash table is a length hash table as the attached information of the interval. The key of the hash table is the fingerprint length, and the value is the second-level hash table (i.e. the fingerprint hash table). The second level is a fingerprint hash table, the key is the binary content of the fingerprint, and the value is the configuration information of the fingerprint. In order to allow the input data to find as many matching fingerprints at once as possible, it is necessary to know the length of the longest fingerprint in the same offset position fingerprint, and therefore a longest fingerprint length is added at the node. This forms the data structure of the nodes of the interval index tree.
Step 1012: and establishing an interval index empty tree, and if fingerprints need to be added during initialization, introducing a fingerprint adding sub-module to add the fingerprints.
(2) Fingerprint adding submodule
The fingerprint adding submodule is responsible for adding fingerprints to the interval index tree and mainly comprises the following steps:
step 1021: and searching the interval index tree according to the offset position of the fingerprint, if no interval node with the same offset position exists, creating an interval node according to the step 1011, creating a two-stage hash table, and adding the current fingerprint information into the hash table. If so, step 1022 is entered.
Step 1022: and searching the two-stage hash table to inquire whether the current node has fingerprints with the same length. If not, step 1024 is entered. If there are fingerprints of the same length and different binary contents, step 1023 is entered. If there is a fingerprint with the length same as the binary content of the fingerprint, the fingerprint is prohibited from being added, and the fingerprint adding is finished.
Step 1023: and inserting data into the fingerprint hash table. And adding the binary content of the fingerprint and the fingerprint configuration information into the fingerprint hash table. The fingerprint addition is ended. The fingerprint in the invention can refer to audio and video characteristics extracted by machine learning, neural network and the like, and the fingerprint configuration information refers to audio and video information identified by the corresponding fingerprint, such as whether piracy exists or not and whether yellow storm exists or not.
Step 1024: and creating a fingerprint hash table, adding the binary content of the fingerprint and the fingerprint configuration information into the fingerprint hash table, and simultaneously inserting the fingerprint length and the just created fingerprint hash table handle into the length hash table. And simultaneously updating the length of the longest fingerprint in the interval and finishing fingerprint addition.
(3) Fingerprint deleting submodule
The fingerprint deleting submodule is responsible for deleting the designated fingerprints in the interval index tree and mainly comprises the following steps:
step 1031: searching interval nodes with the same offset position, if not, returning to the invalid deletion without fingerprints needing to be deleted, and finishing the deletion. If so, go to step 1032.
Step 1032: and searching the length hash table in the secondary hash table of the current node, if no fingerprint hash table handle with the corresponding length exists, indicating that no fingerprint with the length exists, deleting the invalid fingerprint, and finishing deleting. If so, step 1033 is entered.
Step 1033: and searching the fingerprint hash table by using the binary content of the fingerprint, if the corresponding fingerprint configuration information does not exist, the corresponding fingerprint does not exist, deleting invalidity is realized, and deleting is finished. If so, go to step 1034.
Step 1034: and deleting the corresponding fingerprint binary fingerprint and the fingerprint configuration information in the fingerprint hash table. Check if the fingerprint hash table is empty. If not, the deletion is ended. If empty, step 1035 is entered.
Step 1035: and deleting the corresponding fingerprint length and the fingerprint hash table handle in the length hash table, and updating the length of the longest fingerprint. Check if the length hash table is empty. If not, the deletion is ended. If the node is empty, the node indicates that no fingerprint exists in the interval node, the node is deleted, and the deletion is finished.
2. Fingerprint identification module
Step 2001: and (4) streaming inputting the audio and video content, and ending the matching process if the input is ended. Otherwise, if the last matching residual cache region is not empty, merging the residual cache region with the current input data to serve as a data block to be matched, wherein the offset position of the data block is the offset position of the cache region; otherwise (the last matching residual buffer area is empty), the current input is directly used as the data block to be matched.
Step 2002: in order to be able to query the corresponding fingerprint configuration information using the two-level hash table, the data needs to completely contain the fingerprint (as incase 1 of fig. 3). Therefore, first, the section index tree is searched to see whether or not a section having a length of one among the section nodes is included in the input data (since the section length is 1, there is no case of partial overlapping). If not, the data is discarded. If so, step 2003 is entered.
Step 2003: whether the longest fingerprint in the node of the section is included in the data is determined by the longest fingerprint length of the included section, i.e.case 1 of fig. 3. If notcase 1 of fig. 3 but case 2-4 of fig. 3, the data is stored in the matching residual buffer, and step 2001 is performed to wait for data input. If it iscase 1 of fig. 3, then step 2004 is entered.
Step 2004: and traversing keys of the length hash table, and finding the fingerprint hash tables of all fingerprints in the interval nodes through the length hash table. And taking a section of data with the same offset position and length as the fingerprint in the data as a key of the fingerprint hash table to obtain a corresponding value. If value does not exist, the data does not match the fingerprint, ending the match. If value exists, the data is successfully matched with the fingerprint, and the success state and the fingerprint configuration information are output.
In fig. 3,case 1 is that the input data completely contains a fingerprint interval; case 2 is where the input data partially coincides with the fingerprint interval, and there is a portion of the fingerprint interval before the initial offset position of the input data; case 3 is that the fingerprint interval contains input data completely; case 4 is where the input data partially coincides with the fingerprint section and there is a portion of the fingerprint section after the offset position at the end of the input data.
As shown in fig. 2, for cases 2, 3, and 4, data merging is required, and the process of data merging includes:
1. merging the data input each time with the data in the matching residual cache region;
2. searching an interval index tree, and checking whether the longest fingerprint of a certain offset position is completely contained in the merged data;
3. if not, the merged data is put into the matching residual cache region, and the next data input is waited.
Some explanation is made here on data merging, because data is input sequentially from beginning to end, in the relation between input data and fingerprints, what appears first must becase 1 and case 4 of fig. 3. Then cases 2 and 3 will occur.
The following are two methods of data consolidation intocase 1 of fig. 3.
1) Fig. 3 case 4+ n fig. 3 case 3+ fig. 3 case 2: the data input in sequence are respectively as in the case 4 of fig. 3, the case 3 of fig. 3 and the case 2 of fig. 3, the data are combined step by step, the relationship between the combined data and the fingerprint is as in thecase 1 of fig. 3, and n represents a plurality of data.
2) Fig. 3 case 4+ fig. 3 case 2: the data input in sequence are merged as shown in the case 4 of fig. 3 and the case 2 of fig. 3, and the relationship between the merged data and the fingerprint is shown in thecase 1 of fig. 3.
Experimental data: the data is the real traffic of a certain company gateway in 24 hours, 7655 audio and video files are collected, the audio and video features are extracted, and the mapping relation of the audio and video features and the audio and video files is established. By inputting the content of the audio and video files into the system and carrying out matching identification according to the audio and video characteristics, the matching accuracy is 92.3 percent, the recall rate is 90.1 percent, and the actual application requirements are totally met.
Based on the same inventive concept, another embodiment of the present invention provides an electronic device (computer, server, smartphone, etc.) comprising a memory storing a computer program configured to be executed by the processor and a processor, the computer program comprising instructions for performing the steps of the inventive method.
Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium (e.g., ROM/RAM, magnetic disk, optical disk) storing a computer program, which when executed by a computer, performs the steps of the inventive method.
The foregoing disclosure of the specific embodiments of the present invention and the accompanying drawings is directed to an understanding of the present invention and its implementation, and it will be appreciated by those skilled in the art that various alternatives, modifications, and variations may be made without departing from the spirit and scope of the invention. The present invention should not be limited to the disclosure of the embodiments and drawings in the specification, and the scope of the present invention is defined by the scope of the claims.
Claims (10)
1. A stream matching method based on audio and video dynamic characteristics is characterized by comprising the following steps:
establishing an interval index tree for all fingerprint intervals by taking the offset position as a standard, and storing fingerprint information of the audio and video by using the interval index tree;
and for the data block of the audio and video content to be matched, searching the interval index tree to find out the matched fingerprint and outputting a fingerprint matching result, thereby realizing audio and video identification.
2. The method of claim 2, wherein the interval index tree assigns fingerprints with different offset positions to different intervals, and places fingerprints with the same offset position in the same interval; the interval index tree comprises a plurality of interval nodes, each interval node comprises an interval with the length of 1 and interval attached information, and the interval attached information comprises the length of the longest fingerprint and a first-level hash table; the first-stage hash table is a length hash table, the key of the first-stage hash table is the fingerprint length, and the value of the first-stage hash table is a second-stage hash table; the second-level hash table is a fingerprint hash table, the key of the second-level hash table is the binary content of the fingerprint, and the value is the configuration information of the fingerprint.
3. The method of claim 2, wherein the fingerprint is added to the interval index tree using the steps of:
step 1021: searching the interval index tree according to the offset position of the fingerprint, if no interval node with the same offset position exists, establishing an interval node, establishing a two-stage hash table, and adding the current fingerprint information into the hash table; if there is an interval node with the same offset position, go to step 1022;
step 1022: searching two-stage hash tables to inquire whether the current node has fingerprints with the same length; if not, go to step 1024; if there are fingerprints with the same length and different binary contents, go to step 1023; if the fingerprint with the same length as the binary content of the fingerprint exists, the fingerprint is forbidden to be added, and the fingerprint adding is finished;
step 1023: inserting data into the fingerprint hash table, adding fingerprint binary content and fingerprint configuration information into the fingerprint hash table, and finishing fingerprint addition;
step 1024: creating a fingerprint hash table, adding the binary content of the fingerprint and the fingerprint configuration information into the fingerprint hash table, and simultaneously inserting the fingerprint length and the handle of the fingerprint hash table just created into the length hash table; and simultaneously updating the length of the longest fingerprint in the interval and finishing fingerprint addition.
4. The method of claim 2, wherein the specified fingerprint in the interval index tree is deleted by:
step 1031: searching interval nodes with the same offset position, if not, returning to the invalid deletion and finishing the deletion, wherein the fingerprint which needs to be deleted does not exist; if yes, go to step 1032;
step 1032: searching a length hash table in a secondary hash table of the current node, if no fingerprint hash table handle with the corresponding length exists, indicating that no fingerprint with the length exists, deleting ineffectively, and finishing deleting; if so, go to step 1033;
step 1033: retrieving the fingerprint hash table by using the binary content of the fingerprint, if no corresponding fingerprint configuration information exists, then no corresponding fingerprint exists, deleting invalidity, and finishing deleting; if so, go to step 1034;
step 1034: deleting the corresponding fingerprint binary fingerprint and fingerprint configuration information in the fingerprint hash table, and checking whether the fingerprint hash table is empty; if not, finishing the deletion; if empty, go to step 1035;
step 1035: deleting the corresponding fingerprint length and the fingerprint hash table handle in the length hash table, updating the length of the longest fingerprint, and checking whether the length hash table is empty; if not, finishing deleting; if the node is empty, the node indicates that no fingerprint exists in the interval node, the node is deleted, and the deletion is finished.
5. The method according to claim 2, wherein for the data block of the audio-video content to be matched, finding the matched fingerprint by searching the interval index tree and outputting the fingerprint matching result, comprises:
step 2001: streaming input audio and video content, and ending the matching process if the input is ended; otherwise, if the last matching residual cache region is not empty, merging the residual cache region with the current input data to serve as a data block to be matched, wherein the offset position of the data block is the offset position of the cache region; if the last matching residual cache area is empty, directly taking the current input as a data block to be matched;
step 2002: searching an interval index tree, and checking whether an interval with the length of 1 in interval nodes is included in the input data; if not, discarding the data; if yes, go to step 2003;
step 2003: judging whether the longest fingerprint in the interval nodes is contained in the data or not according to the length of the longest fingerprint of the contained interval nodes, namely the input data completely contains the fingerprint interval; if not, storing the data in the matching residual cache region, and entering step 2001 to wait for data input; if so, go to step 2004;
step 2004: traversing keys of the length hash table, finding fingerprint hash tables of all fingerprints in the interval nodes through the length hash table, and taking a section of data with the same offset position and length as the fingerprints in the data as the keys of the fingerprint hash tables to obtain a corresponding value; if the value does not exist, the data is not matched with the fingerprint, and the matching is finished; if value exists, the data is successfully matched with the fingerprint, and the success state and the fingerprint configuration information are output.
6. The method of claim 5, wherein the step 2001 includes 4 cases of overlapping between the input data and the fingerprint region when performing data merging: case 1 is that the input data contains a fingerprint interval completely; case 2 is where the input data partially coincides with the fingerprint interval, and there is a portion of the fingerprint interval before the initial offset position of the input data; case 3 is that the fingerprint interval contains input data completely; case 4 is where the input data partially coincides with the fingerprint section and there is a portion of the fingerprint section after the offset position at the end of the input data.
7. The method of claim 6, wherein the data is merged into case 1 in two ways:
1) case 4+ n case 3+ case 2: the method comprises the steps of gradually merging data when sequentially input data are respectively in a case 4, a case 3 and a case 2, the relation between the merged data and fingerprints is a case 1, and n represents a plurality of data;
2) case 4+ case 2: the data is merged when the data sequentially input is case 4 and case 2, respectively, and the relationship between the merged data and the fingerprint is case 1.
8. A stream matching device based on audio-video dynamic characteristics by adopting the method of any one of claims 1 to 7, which is characterized by comprising:
the fingerprint management module is used for establishing an interval index tree for all fingerprint intervals by taking the offset position as a standard and storing the fingerprint information of the audio and video by using the interval index tree;
and the fingerprint identification module is used for searching the interval index tree for the matched fingerprint of the data block of the audio and video content to be matched and outputting a fingerprint matching result, so that the audio and video identification is realized.
9. An electronic apparatus, comprising a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program comprising instructions for performing the method of any of claims 1 to 7.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program which, when executed by a computer, implements the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010987148.4ACN112347272B (en) | 2020-09-18 | 2020-09-18 | A streaming matching method and device based on dynamic features of audio and video |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010987148.4ACN112347272B (en) | 2020-09-18 | 2020-09-18 | A streaming matching method and device based on dynamic features of audio and video |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112347272Atrue CN112347272A (en) | 2021-02-09 |
| CN112347272B CN112347272B (en) | 2023-03-10 |
Family
ID=74357955
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010987148.4AActiveCN112347272B (en) | 2020-09-18 | 2020-09-18 | A streaming matching method and device based on dynamic features of audio and video |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112347272B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113495901A (en)* | 2021-04-20 | 2021-10-12 | 河海大学 | Variable-length data block oriented quick retrieval method |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110022638A1 (en)* | 2007-06-27 | 2011-01-27 | Wenyu Jiang | Incremental Construction of Search Tree with Signature Pointers for Identification of Multimedia Content |
| CN102314875A (en)* | 2011-08-01 | 2012-01-11 | 北京百度网讯科技有限公司 | Audio file identification method and device |
| US20120278326A1 (en)* | 2009-12-22 | 2012-11-01 | Dolby Laboratories Licensing Corporation | Method to Dynamically Design and Configure Multimedia Fingerprint Databases |
| CN103093761A (en)* | 2011-11-01 | 2013-05-08 | 腾讯科技（深圳）有限公司 | Audio fingerprint retrieval method and retrieval device |
| CN104765836A (en)* | 2015-04-15 | 2015-07-08 | 王磊 | Multi-level index method for data of audio fingerprint database |
| US20180285455A1 (en)* | 2017-03-31 | 2018-10-04 | Gracenote, Inc. | Multiple stage indexing of audio content |
| CN109388729A (en)* | 2017-08-14 | 2019-02-26 | 阿里巴巴集团控股有限公司 | Search method, device and the audio query system of audio sub fingerprint |
- 2020
- 2020-09-18CNCN202010987148.4Apatent/CN112347272B/enactiveActive
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110022638A1 (en)* | 2007-06-27 | 2011-01-27 | Wenyu Jiang | Incremental Construction of Search Tree with Signature Pointers for Identification of Multimedia Content |
| US20120278326A1 (en)* | 2009-12-22 | 2012-11-01 | Dolby Laboratories Licensing Corporation | Method to Dynamically Design and Configure Multimedia Fingerprint Databases |
| CN102314875A (en)* | 2011-08-01 | 2012-01-11 | 北京百度网讯科技有限公司 | Audio file identification method and device |
| CN103093761A (en)* | 2011-11-01 | 2013-05-08 | 腾讯科技（深圳）有限公司 | Audio fingerprint retrieval method and retrieval device |
| CN104765836A (en)* | 2015-04-15 | 2015-07-08 | 王磊 | Multi-level index method for data of audio fingerprint database |
| US20180285455A1 (en)* | 2017-03-31 | 2018-10-04 | Gracenote, Inc. | Multiple stage indexing of audio content |
| CN109388729A (en)* | 2017-08-14 | 2019-02-26 | 阿里巴巴集团控股有限公司 | Search method, device and the audio query system of audio sub fingerprint |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113495901A (en)* | 2021-04-20 | 2021-10-12 | 河海大学 | Variable-length data block oriented quick retrieval method |
| CN113495901B (en)* | 2021-04-20 | 2023-10-13 | 河海大学 | Quick retrieval method for variable-length data blocks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112347272B (en) | 2023-03-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10198427B2 (en) | System and method for keyword spotting using representative dictionary | |
| CN103164698B (en) | Text fingerprints library generating method and device, text fingerprints matching process and device | |
| CN101826099B (en) | Method and system for identifying similar documents and determining document diffusance | |
| GB2509773A (en) | Automatic genre determination of web content | |
| US20190377764A1 (en) | Illegal content search system and method thereof | |
| CN108875366A (en) | A kind of SQL injection behavioral value system towards PHP program | |
| Harichandran et al. | Bytewise approximate matching: the good, the bad, and the unknown | |
| KR20120112696A (en) | Malware detection via reputation system | |
| CN112989348B (en) | Attack detection method, model training method, device, server and storage medium | |
| CN108959359B (en) | Uniform Resource Locator (URL) semantic deduplication method, device, equipment and medium | |
| CN105024987B (en) | A kind of monitoring method and device of web business diaries | |
| CN111740946A (en) | Webshell message detection method and device | |
| US8463725B2 (en) | Method for analyzing a multimedia content, corresponding computer program product and analysis device | |
| CN118337453A (en) | Automatic attack tracing method, terminal device and storage medium | |
| CN112347272A (en) | A method and device for streaming matching based on dynamic features of audio and video | |
| CN114154043B (en) | Website fingerprint calculation method, system, storage medium and terminal | |
| WO2025130497A1 (en) | Method, apparatus and device for identifying network attack | |
| KR101545273B1 (en) | Apparaus and method for detecting dupulicated document of big data text using clustering and hashing | |
| US10025936B2 (en) | Systems and methods for SQL value evaluation to detect evaluation flaws | |
| CN113821630A (en) | Data clustering method and device | |
| CN107077509B (en) | Method, device and equipment for improving browsing experience | |
| CN115392238A (en) | Equipment identification method, device, equipment and readable storage medium | |
| CN115209183B (en) | Domain name association method for video resources and video playback pages for encrypted traffic | |
| JP5464082B2 (en) | Document processing apparatus, document processing method, document processing program, and computer-readable recording medium recording the document processing program | |
| CN115859380B (en) | Electronic data fixed certificate association method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |