Disclosure of Invention
The invention aims to provide an electronic screen safety verification system based on a dynamic screen refresh rate and real-time watermarking, which is used for monitoring the legitimacy of electronic screen playing video content, automatically identifying and blocking the playing of abnormal content and preventing illegal spreading of the bad content by constructing an electronic screen playing content tampering detection method with higher safety and low delay detection.
In order to achieve the above purpose, the invention provides the technical scheme that the electronic screen safety verification system based on the dynamic screen refresh rate and the real-time watermark is characterized in that a plurality of network electronic screens are connected with a cloud server through a network, and a safety monitoring terminal is communicated with the cloud server through network connection;
The security monitoring terminal transmits the acquired network electronic screen equipment information and equipment codes of the security monitoring terminal to the cloud server in a TLS encrypted communication mode;
the cloud server embeds a dynamic watermark into a video to be played of the network electronic screen, and the video embedded with the watermark is obtained after watermark modulation;
The security monitoring terminal requests the cloud server to acquire a watermark time sequence coding sequence of the video to be played through TLS encryption communication, and the network electronic screen terminal downloads the video to be played and plays the video;
the camera module of the safety monitoring terminal captures a screen refreshing time sequence;
and performing video playing abnormality detection judgment and response.
In some embodiments, the network electronic screen device information includes a network electronic screen refresh frequency, a network electronic screen device hardware serial number.
In some embodiments, the embedding the dynamic watermark comprises:
generating a watermark basic sequence, including a serial number of a network electronic screen device and the serial number of a current video;
generating a watermark timing encoding sequence based on the watermark base sequence;
dividing each video frame into a plurality of refresh cycles;
In the embedded region pairAnd (5) watermark modulation is carried out on the brightness of the video pixel at the moment, so that the video embedded with the watermark is obtained.
In some embodiments, the computational expression for watermark modulating the video pixel luminance at time t in the embedded region is:
;
Wherein, the
;
In the formula,The brightness value of the pixel after modulation; For the original videoTime of day imageAn original luminance value at; for modulating amplitude; elements in the watermark timing sequence are used to determine the positive and negative modulation of the pixel,,A time sequence coding sequence for the watermark; is an empirical constant; Is the average luminance of the local area; global maximum and minimum, respectively.
In some embodiments, the security monitoring terminal requests the cloud server to obtain a watermark timing encoding sequence of a video to be played through TLS encrypted communication, including:
Collecting a screen picture, and calculating the average brightness of a watermark embedding area to be used as a timing sequence signal for screen refreshing;
extracting a periodic signal of the time sequence signal and performing Fourier transformation;
Decoding the watermark sequence, and performing coherent adjustment on the periodic signal;
and calculating and outputting the error rate.
In some embodiments, the computational expression for coherent conditioning of the periodic signal is:
Wherein the method comprises the steps of
In the formula,For the duration of the period of time,;Is the screen refresh rate; For the index of the refresh period,;As a sign function.
In some embodiments, the security monitoring terminal performs anomaly detection determination and response on a video played by the network electronic screen, including:
determining whether the current playing is abnormal or not based on the refresh rate deviation degree;
determining whether the current playing is abnormal or not based on the error rate tolerance of the time sequence coding;
And setting a play abnormality detection response mechanism, namely judging that the currently played content is tampered when the continuous multi-frame video is abnormal or the accumulated abnormal times exceeds a threshold value in a fixed time, and shutting down the network electronic screen.
Compared with the prior art, the invention has the beneficial effects that:
1. The invention has stronger anti-attack capability that the screen refreshing frequency is dynamic and non-fixed, and the static screen capturing or common screen recording can not capture the physical refreshing time sequence (such as the periodic brightness fluctuation of 78 Hz) of the screen. The watermark sequence is bound with the dynamic screen refresh rate, the equipment serial number and the video number, the dynamic watermark has uniqueness, and even if an attacker acquires video content, the attacker cannot reproduce the specific dynamic watermark time sequence.
2. The invention has real-time property in detecting low delay, wherein the single check period isFor example, under the 60Hz screen refresh rate, the single check time is 16.7ms, and the detection is carried out every 16.7ms, so that the detection has low delay and real-time performance.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1-3, an electronic screen security verification system based on a dynamic screen refresh rate and a real-time watermark comprises a cloud server, a network electronic screen and a security monitoring terminal, wherein the security monitoring terminal is a core execution unit of a real-time watermark embedding and verification scheme and is responsible for dynamically controlling the screen refresh rate, capturing screen signals, detecting anomalies and triggering a blocking mechanism. Comprising the following steps:
And the camera module is characterized in that the core hardware is a camera, the frame rate of more than 240Hz is supported, and an IMX477 camera can be adopted.
And the electronic screen dynamic frequency setting module adopts a control chip supporting HDMI 2.1 VRR protocol, such as STM32H743 microcontroller, and dynamically sets the refreshing frequency (such as 78Hz, 90Hz and other arbitrary values) of the network electronic screen according to the watermark embedding requirement through the control chip.
The anomaly detection module is used for processing the brightness signal input by the camera module in real time, carrying out periodic signal analysis, watermark sequence decoding and anomaly judgment, synchronizing data to the cloud server, and enabling the processor to support multi-thread parallel processing, wherein the processor model is T507×A53@1.8GHz+G52Mp2.
The electronic screen power blocking module adopts a solid-state relay for cutting off the power supply of the screen after detecting the abnormality, and the relay type number can be G5V-2-H1 DC5V solid-state relay.
The four modules of the safety monitoring terminal work cooperatively through a hardware interface, a data bus and a control signal line to form a closed-loop control system.
The network electronic screens are connected with the cloud server through a network and used for acquiring video content to be played, the network electronic screens are connected with the safety monitoring terminal, and the network electronic screens are displays supporting variable refresh rate VRR protocols (such as FreeXync and G-Sync).
And the safety monitoring terminal is communicated with the cloud server through network connection.
The safety verification system realizes verification by the following method:
S1, a safety monitoring terminal acquires information of electronic screen equipment connected with the safety monitoring terminal, randomly sets refreshing frequency of a network electronic screen within a certain range (for example, 60-120 Hz), and sends hardware serial numbers and refreshing frequency of the network electronic screen equipment to the safety monitoring terminalAnd transmitting the information and the information such as the serial numbers of the safety monitoring terminal equipment to a cloud server side through TLS encrypted communication.
S2, the cloud server embeds a dynamic watermark into the video to be played on the network electronic screen.
The original video frame sequence is noted asScreen refresh rate of network electronic screen displayOriginal video frame rate. The method comprises the following specific steps:
(1) A watermark base sequence is generated.
Obtaining a unique serial number SID of a network electronic screen device, obtaining a serial number id of a current video, converting a result obtained by calculating the serial number SID and the serial number id of the current video through a cryptographic hash function SHA256 (SID, id) into a binary sequence to obtain a dynamic watermark basic sequence。
(2) Generating watermark timing encoding sequence based on watermark base sequence B
In the formula (I), in the formula (II),Is a new element with the value of 1 or-1 after the binary element of the watermark basic sequence is converted; is the kth element in the base sequence generated in step (1).
(3) Dividing a number of refresh cycles for each video frameFor example, the first and second substrates may be coated, for example,Each video frame is divided intoAnd a refresh period.
(4) Watermark modulation in specific embedded regionsFor example, four corners of the screen edgeWatermark modulation is carried out on video pixel brightness at moment:
;
Wherein, the
;
In the formula,Elements in the watermark timing sequence are used to determine the positive and negative modulation of the pixel,,A sequence is encoded for watermark timing.
For each watermark modulation, time sequence coding elements are obtained from a watermark time sequence coding sequence according to the sequence order and used as the basis of watermark information to refresh the cycle numberFor example, during a first refresh period (start and end times of 0, respectively),) Pixel brightness modulation watermark informationIn the second refresh period (start and end times are respectively、) Pixel brightness modulated watermark informationSimilarly, when the watermark timing code sequence is obtainedAt the last element, the watermark is time-sequence encodedAnd (5) recycling and reusing. Obtaining a video embedded with watermark after watermark modulation, wherein the watermark is recorded as a sequence。
S3, the security monitoring terminal requests the cloud server to acquire a watermark time sequence coding sequence of the video to be played through TLS encrypted communicationThe network electronic screen terminal downloads the video to be played and plays the video;
S4, capturing a refreshing time sequence of a network electronic screen by a camera module of the safety monitoring terminal, wherein the refreshing time sequence comprises the following steps of:
s41, acquiring a screen picture through a camera module of the safety monitoring terminal, wherein the frame rate of the camera moduleCalculating watermark embedding area in screen acquisition pictureAs a timing signal for screen refresh:。
Acquired for camera moduleThe instantaneous brightness value of the moment in time,For averaging functions; Is the image of the original video at the moment tAt the original luminance value of the light source,Representing pixel position coordinates of the image.
S42, extracting periodic signals of the time sequence signals, forFourier transforming to extract fundamental frequencyThe components are as follows: in which, in the process,Peak frequency as fourier transformWhereinThe frequency at the corresponding position is the peak frequency。
S43, decoding the watermark sequence, and performing coherent adjustment on the periodic signal:
;
Wherein, the
;
In the formula,A decoded watermark timing sequence; for the duration of the period of time,;Is the screen refresh rate; For the index of the refresh period,;As a sign function.
S44, calculating and outputting the error rate.
Calculating an actual decoded sequenceTiming with watermarkError rate of code sequence,As an indicative function, the output is 1 when the condition is satisfied, otherwise the output is 0.
S5, detecting, judging and responding to video playing abnormality. The method comprises the following steps:
Timing offset abnormality determination, determining whether or not the current play is abnormal based on the degree of the refresh rate deviation, when the detected refresh rate deviation exceeds a threshold value, that isTime%2Hz for the threshold value), and the current playing abnormality is determined.
Determining whether the current playing is abnormal based on the error rate tolerance of the time sequence coding, and when the error rate of the time sequence coding exceeds the tolerance, namelyTime%For the threshold value, 0.05 is preferable), and the current playing abnormality is determined.
And setting a play abnormality detection response mechanism, namely judging that the current play content is tampered when abnormality occurs in continuous multi-frame (e.g. 5 frames) video or the accumulated abnormality times are excessive (e.g. the abnormality times are more than 10 times in 3 seconds) in a short time, reporting report information of illegal play to a cloud server, and cutting off a screen of a network electronic screen through a power relay to supply power to stop the transmission of illegal content.
In a specific embodiment, the refresh rate of the screen of the network electronic screen is set to be 60Hz and 120Hz, and the watermark embedding efficiency test of the embodiment is performed by taking full-high-definition and ultra-high-definition videos as original videos. The real-time test is verified by using the normal video playing, fast moving pictures and strong ambient light interference using scene test of the embodiment, and the attack resistance test and comparison are carried out by taking screen capturing substitution attack, same screen replay attack and analog watermark attack as attack types, wherein the specific embodiment scheme and the result are as follows:
1. Hardware device parameters are shown in the following table:
table 1 test hardware device parameter table
2. And (5) watermark embedding efficiency test.
The same video to be played has watermark capacity depending on refresh frequency of electronic screenThe size of the watermark capacity determines the total duration of video watermark embedding. The watermark capacity size is: , wherein,In order to number of refresh cycles,The original video frame rate and the total duration are respectively, the video frame rate is 30, the video with the length of 10 seconds, the total video frame number is 300, and the refresh frequency isThe watermark capacity size is 2 x 30 x 10=600 bits.
According to the method, the device serial number and the video number to be played are converted into binary sequences to obtain a dynamic watermark basic sequence through the result obtained by calculating the encryption hash functionThe sequence length is 256, and the sequence is converted into a watermark time sequence coding sequence。
Each refresh period of the video frames is assigned an element in the watermark sequence, 1 st frame is assigned-1 in period 1, 1 in period 2 is assigned 1 in period 2, 1 in period 1 is assigned-1 in period 2, and so on, until each period of 300 video frames is assigned a watermark sequence (when the element in the watermark sequence is insufficient, the watermark sequence is recycled from the initial cycle of the sequence).
Watermark is embedded in four corners of the edge of the screen, namely watermark modulation is carried out on the brightness of the video pixel according to the distributed watermark sequence elements, the video embedded with the watermark is obtained after watermark modulation, and the test result is shown in the following table:
Table 2 watermark embedding efficiency test results table
3. And (5) checking the real-time performance.
In the real-time performance verification, the original video frame rate is 30, the network electronic screen terminal downloads the video to be played and replaces the video with other videos with the same frame rate for playing, a camera module of the safety monitoring terminal captures a screen refreshing time sequence, video playing abnormality detection is carried out, when continuous 5 frames of video are abnormal, abnormal playing is judged when the conditions are met, and the test results are shown in the following table:
Table 3 real time performance test results table
4. Test of anti-attack ability
In the anti-attack capability test, a traditional DCT frequency domain feature code embedding (H.264 coding, embedding complexity and the like) is selected as a comparison scheme, and the anti-attack capability test and comparison are carried out by taking screen capturing substitution attack, same screen replay attack and analog watermark attack as attack types, wherein the results are shown in the following table:
Table 4 the test result table of the present invention and the DCT frequency domain feature code for anti-attack ability
The invention is immune to screen capture replacement attacks (screen capture attacks cannot capture the refreshing time sequence of a screen depending on the physical refreshing characteristics of the screen). For the display of the same type of multiple wrong screens, the watermark information is bound with the unique serial number of the equipment, and the same type of screen supports the VRR protocol with variable refresh rate. For the analog watermark, the watermark information is obtained by hash encryption of the variable dynamic screen refresh rate, the equipment serial number and the video number, the watermark information of any two equipment and the watermark information of the same equipment at different moments are also different, the possibility of simulating the same watermark by watermark cracking is almost zero, and the attack resistance of the invention is effectively improved.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.