Disclosure of Invention
In order to overcome the problems in the prior art, the invention provides an on-line technical intra-examination and comparison method based on real-time signals. The method is used for carrying out multi-dimensional detection on the real-time signals, arbitrating the detection results, comprehensively judging the final-stage signals according to the arbitration results, and selecting the optimal path in the final-stage signals to ensure that the final PGM output is the optimal selection.
The purpose of the invention is realized in the following way: a method for on-line intra-review and comparison based on real-time signals, the method comprising the use of a system comprising: the intelligent fingerprint analysis system comprises an input module capable of receiving various information sources, wherein the input module is connected with a video buffer module, the video buffer module is connected with a fingerprint extraction module, an internal examination module with an AI intelligent analysis library and an online examination module, the fingerprint extraction module is connected with a fingerprint comparison module, the internal examination module and the online examination module are connected with an arbitration module, the arbitration module is connected with an output module, and the method comprises the following steps:
step 1, receiving video and audio data: the input module receives audited video and audio data in various formats, and comprises a main video signal, a standby video signal, a three-standby video signal and two-way video signals of the same video and audio data;
the following steps 2-4 are performed simultaneously;
Step 2, online technical examination: the on-line technical module carries out real-time frame-by-frame detection on the real-time signals, extracts characteristics of the collected video and audio buffers according to the category of the detection items, calculates indexes of the corresponding detection items, judges whether the signals accord with quality inspection standards according to the indexes, carries out real-time collection on each pixel point, carries out detection on signal faults, and realizes real-time on-line technical inspection;
The current path signal is scored according to the result of the online technical examination, and the result is classified into three grades, specifically as follows:
1) Common fault static frame, mute, front Zhou Yin faults are 1 minute;
2) The fault black field, the color field, the black field and silence of the severe faults, the color field and silence fault is 2 minutes;
3) The most serious failure: video loss and color bar fault are 3 minutes;
Step 3, internal examination: the internal examination module calls an AI intelligent analysis library to check the real-time signals of the broadcasting and television level frame by frame, check the real-time signals with sensitive characters in the AI library, alarm after the verification is successful, and perform real-time check analysis on the yellow-related and blood fishy violence content existing in the real-time signals, and alarm in real time when the yellow-related and blood fishy violence content of the signals is detected; the current path signal is scored according to the AI internal examination result, and the result is classified into three grades: the violence of the fishy smell of blood is 1 minute, the sensitive person is 2 minutes, and the yellow-related content is 3 minutes;
Step 4, comparison: the real-time comparison of the two sets of final signals of the main and the standby, the three and the standby is carried out, and the specific flow is as follows:
1) After the signal data are acquired, a fingerprint extraction module is adopted to extract fingerprints, and the fingerprints of the two paths of signals are submitted to a bottom layer after the fingerprints are extracted;
2) After the fingerprint information is obtained, matching the fingerprints of the two paths of signals in an asynchronous working mode, and dividing the steps into two steps, firstly aligning time frames, comparing after aligning the frames, and if the number of the subsequent unmatched frames exceeds the tolerance, realigning and comparing again after aligning;
3) Detecting and calculating the brightness of the signal:
a. Respectively calculating the similarity of Y, U, V of the two paths of signals;
b. Calculating the standard deviation of the ratio of the two paths Y, U, V;
c. sorting the fingerprints of the histograms from small to large, and then calculating the similarity;
d. obtaining a Y similarity threshold according to the brightness integral difference threshold;
e. if the Y similarity is lower than the threshold, the standard deviation is lower than the threshold, U, V, and if the histogram similarity is higher than the threshold, only the brightness is considered to be the whole difference;
4) Calculating a similarity threshold value of the two paths of signals according to the brightness and the similarity, and if the similarity threshold value exceeds a set threshold value, carrying out inconsistent prompt;
step 5, arbitration: arbitrating technical, internal examination and comparison results, analyzing the quality of each path of signal in real time, scoring each path of signal, and obtaining a final result according to the scoring result, wherein the specific calculation result is as follows:
1) The scoring result of the online technical examination is A; the score result of the internal examination is B; the comparison score result is C;
2) The weight proportion of the technical examination is 50%; the proportion of the internal examination is 30 percent; the proportion of the comparison is 20%;
3) Final overall score m=50% > a+30% > b+20% c;
step 6, outputting: and taking the picture with the highest score as an output signal, outputting the picture through an output module, and ending the examination.
The invention has the advantages and beneficial effects that: the invention carries out on-line technical examination on the real-time signals in a pixel analysis mode, carries out internal examination and compares the video fingerprints by using an AI technology, and the whole service system carries out arbitration by combining detection results of the three, comprehensively judges the final-stage signals according to the arbitration result, selects the optimal path therein, and ensures that the final PGM output is optimal. The invention integrates a plurality of online monitoring modes, greatly reduces the detection cost and improves the working efficiency.
Detailed Description
Embodiment one:
The embodiment is a method for on-line technical in-check and comparison based on real-time signals. The system used in the method is shown in fig. 1. The system used in the method of this embodiment includes: the intelligent monitoring system is characterized by comprising an input module capable of receiving various information sources, wherein the input module is connected with a video buffer module, the video buffer module is connected with a fingerprint extraction module, an internal examination module with an AI intelligent analysis library and an online examination module, the fingerprint extraction module is connected with a fingerprint comparison module, the internal examination module and the online examination module are connected with an arbitration module, and the arbitration module is connected with an output module.
The embodiment is mainly aimed at carrying out online technical examination, internal examination and comparative equivalent online examination on the transmitted final-stage signals in real time. And scoring according to the real-time online technical result, the internal result and the comparison result, comprehensively scoring the obtained scores by using corresponding weights, and arbitrating the scores to ensure that the played signals are correct and have good quality.
The input module should have at least the IO card of SDI, the IO card of IP, have multiple kinds of video data signals that broadcast server or network send such as compressing or not compressing IP network card.
The output module can interface with a broadcast network or a broadcast transmission system (e.g., a terrestrial television transmission station).
The embodiment mainly provides an online technical module, an internal examination module with an AI intelligent analysis library and a comparison module, and three modules are applied to meet the service requirements of detection of image quality, content and the like.
The on-line technical module is mainly used for checking the image quality of the video image, including whether each pixel correctly expresses the image, and the on-line checking is mainly used for checking the content of the picture, such as image sensitive characters, yellow-related characters and the like, and comparing whether the image source is legal or not.
The method in this embodiment includes the following steps, and the flow is shown in fig. 2:
step 1, receiving video and audio data: the input module receives audited video and audio data in various formats, including a main video signal, a standby video signal and a three-standby video signal of the same video and audio data.
The broadcast signal will typically be transmitted in a plurality of formats or definitions for customer selection, and since this embodiment is at the end of the broadcast, it will typically be possible to receive a plurality of formats or definitions for the same program.
The following steps 2-4 are performed simultaneously, namely, technical examination, internal examination and comparison are performed simultaneously, and the multithreading treatment can improve the efficiency.
Step 2, online technical examination: the on-line technical module carries out real-time frame-by-frame detection on the real-time signals, extracts characteristics of the collected video and audio buffers according to the category of the detection items, calculates indexes of the corresponding detection items, judges whether the signals accord with quality inspection standards according to the indexes, collects each pixel point in real time, carries out detection of signal faults, and realizes real-time on-line technical inspection.
After fault detection, scoring the current path signal according to the result of online technical examination, classifying the result into three levels, and quantifying the fault so as to bring a formula into overall judgment for calculation, wherein the method comprises the following steps of:
1) The common faults of static frame, silence and front Zhou Yin faults are 1 minute. Static frames, silence and front Zhou Yin are usually instant image quality degradation, normal broadcasting can be quickly restored, visual impact is small, and therefore failure scores are low.
2) The more serious faults of black field, color field, black field and silence, color field and silence fault are 2 minutes. Black, color, black and mute, color and mute are relatively serious faults, and can be considered as prolonged occurrence of static frames, mute and front Zhou Yin, which usually lasts for a few seconds, but can be recovered, and the visual impact is larger, and a higher score is given.
3) The most serious failure: video loss and color bar failure is 3 minutes. Video loss, color bar failure can be considered as black field, color field, black field and mute, long duration of color field and mute, and cannot be recovered, pause the playing process, of course, the greatest impact on vision, thus giving the highest score.
Step 3, internal examination: the internal examination module calls an AI intelligent analysis library to check the real-time signals of the broadcasting and television level frame by frame, check the real-time signals with sensitive characters in the AI library, alarm after the verification is successful, and perform real-time check analysis on the yellow-related and blood fishy violence content existing in the real-time signals, and alarm in real time when the yellow-related and blood fishy violence content of the signals is detected; the current path signal is scored according to the AI internal examination result, and the result is classified into three grades: the violence of the blood fishy smell is 1 minute, the sensitive person is 2 minutes, and the yellow content is 3 minutes.
The internal examination intelligently adopts the technical route of an AI analysis database, namely, a fault database is constructed, the fault database is continuously learned in practical application, the fault database is supplemented, the using method of fault data is continuously learned, video signals are checked frame by frame, the image data stored in the analysis database are intelligently compared, and the image content is scored.
Step 4, comparison: the real-time comparison of the two sets of final signals of the main and the standby, the three and the standby is carried out, and the specific flow is as follows:
1) In order to realize efficient and stable comparison effect, after signal data are acquired, a fingerprint extraction module is firstly adopted to extract video and audio fingerprints to extract the fingerprints, and then the fingerprints of two paths of signals are submitted to a bottom layer.
2) After the fingerprint information is obtained, the fingerprints of the two paths of signals are matched in an asynchronous working mode, the method is divided into two steps, firstly, frame alignment is carried out, the similarity of the fingerprints of the two paths of signals is calculated after frame-by-frame alignment, comparison is carried out, if the similarity is lower than a set threshold value, the two paths of signals are considered to be unmatched, if the number of continuous subsequent unmatched frames exceeds the tolerance, the two paths of signals are considered to be inconsistent, alarming is carried out, realignment is carried out, and the frame-by-frame comparison is carried out again after alignment.
3) Further detecting whether the two paths of signals only have brightness difference, and detecting and calculating the brightness of the signals:
a. and respectively calculating the similarity of Y, U, V of the two paths of signals.
B. the standard deviation of the ratio of the two paths Y, U, V is calculated.
C. the histogram fingerprints are sorted from small to large and then the similarity is calculated.
D. and obtaining a Y similarity threshold according to the brightness integral difference threshold.
E. If the Y similarity is lower than the threshold, the standard deviation is lower than the threshold (the usable empirical value is 0.03,), and if U, V and the histogram similarity are higher than the threshold, only the brightness is considered to be totally different.
4) And calculating a similarity threshold value of the two paths of signals according to the brightness and the similarity, and if the similarity threshold value exceeds a set threshold value, giving inconsistent prompt.
Step 5, arbitration: arbitrating technical, internal examination and comparison results, analyzing the quality of each path of signal in real time, scoring each path of signal, and obtaining a final result according to the scoring result, wherein the specific calculation result is as follows:
1) The scoring result of the online technical examination is A; the score result of the internal examination is B; the comparison score result is C;
2) The weight proportion of the technical examination is 50%; the proportion of the internal examination is 30 percent; the proportion of the comparison is 20%;
3) Final overall score m=50%a+30%b+20%c;
each path can obtain a final evaluation score according to the formula, and the final score is used as the obtained score of the path, and an optimal picture is selected as an output signal according to the score of each path in the final stage.
Step 6, outputting: and taking the picture with the highest score as an output signal, outputting the picture through an output module, and ending the examination.
Finally, it should be noted that the above only illustrates the technical solution of the present invention, and not limiting, and although the present invention has been described in detail with reference to the preferred arrangement, it will be understood by those skilled in the art that modifications and equivalent substitutions can be made to the technical solution of the present invention (such as the system used, the kind of input or output signals, the sequence of steps, etc.), without departing from the spirit and scope of the technical solution of the present invention.