US20110141372A1 - Video evaluation device, frame rate determination device, video process device, video evaluation method, and video evaluation program - Google Patents

Abstract

A frame rate is determined in accordance with the smoothness in movement of a video. An amount of change detection section extracts an amount of change based on a plurality of frame pictures included in an input video signal input as a moving picture signal from the outside and outputs an amount of change to an evaluation value calculation section. The evaluation value calculation section computes an amount of change in time between each of the frame pictures based on the amount of change and the time interval between frame pictures based on frame rate information and evaluates an evaluation value for evaluating the smoothness in movement of an input video based on the amount of change in time. The evaluation value is output to the outside as a factor for determining a frame rate.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
• This application is a continuation of and is based upon and claims the benefit of priority under 35 U.S.C. §120 for U.S. Ser. No. 11/226,317, filed Sep. 15, 2005, the entire contents of which are incorporated herein by reference and which claims the benefit of priority under 35 U.S.C. §119 from Japanese Patent Application No. 2005-229618, filed Aug. 8, 2005 and Japanese Patent Application No. 2004-270123, filed Sep. 16, 2004.
BACKGROUND OF THE INVENTION
• 1. Field of the Invention
• The present invention relates to video evaluation device, frame rate determination device, video process device, a video evaluation method, and a video evaluation program.
• 2. Related Background of the Invention
• In a video process including acquisition, storing, transmission, displaying, encoding, decoding, etc. of a video, in general, the process is carried out on the basis of a fixed frame rate. Here, the frame rate is the number of frames processed per second. The fixed frame rate is the frame rate at which the number of frames processed per second is constant. As a concrete example of the fixed frame rate is defined as 29.97 fps (frame per second) according to, for example, National Television Standards Committee (NTSC) standards adopted in the United States and Japan. Alternatively, it is defined as 25 fps according to Phase Alternating Line (PAL) standards of National Television Standards Committee adopted in Europe. Further, the fixed frame rate of 15 fps or 24 fps may be used. A “video” is composed of successive “frame pictures”, which are individual still pictures.
• When a video process is carried out at a fixed frame rate, if the frame rate is increased, the time intervals between successive frames become shorter. Due to this, it becomes possible to process a video smoother in movement. For example, a video to be processed at a frame rate of 30 fps has the number of frames per unit time larger than that of a video to be processed at a frame rate of 15 fps, therefore, the movement of the video is represented in a finer manner and a smoother movement is represented as a whole.
• In addition to the video process at the above-mentioned fixed frame rate, a video process at a variable frame rate is also carried out. In the video process at a variable frame rate, the frame rate is varied in accordance with the amount of process of a video and the amount of data. For example, when a video is encoded, if it is judged that the amount of data to be encoded is large, the frame rate is reduced and the number of frames to be encoded per unit time is reduced. This is because if the amount of data increases, the time required for video process is increased. Here, at a variable frame rate, if the time interval between successive frame pictures is assumed to be T, the frame rate between two frame pictures is 1/T.
• A technique to change such a frame rate is disclosed in, for example, Patent document 1 (Japanese Patent Application Laid-open No. Hei 11-112940).
SUMMARY OF THE INVENTION
• By the way, when a video is processed at a fixed frame rate, if the frame rate is increased in order to realize a smooth movement, the amount of process, the amount of data, and the power consumption accompanying the video process increase. This is explained specifically. For example, when a video is acquired, if the number of frames to be acquired per unit time increases, the amount of process and the power consumption accompanying the process increase. Further, when a video is stored, the amount of data increases because the number of frames to be stored per unit time increases.
• On the other hand, if the frame rate is reduced in order to reduce the amount of process, the amount of data, and the power consumption accompanying the process of a video, the smoothness in movement of a video is degraded, resulting in a video with unsmooth movement.
• When a video is processed at a variable frame rate, if the frame rate is changed in accordance with only the amount of process and the amount of data of the video, the smoothness in movement of the video is degraded, resulting in a video with unsmooth movement.
• As described above, if the frame rate is changed without taking into consideration the feature of movement of a video, there arises a problem that the amount of process, the amount of data, and the power consumption are required more than necessary and that a video with unsmooth movement results.
• In order to solve the above-mentioned problem, an object of the present invention is to provide video evaluation device, frame rate determination device, video process device, a video evaluation method, a frame rate determination method, a video process method, a video evaluation program, a frame rate determination program, and a video process program for determining a frame rate in accordance with the smoothness in movement of a video.
• The video evaluation device of the present invention is characterized by comprising an amount of change detection means for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in an input video signal and an evaluation value calculation means for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture and outputting it to the outside.
• The video evaluation method of the present invention is characterized by comprising an amount of change detection step for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in an input video signal and an evaluation value calculation step for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture and outputting it to the outside.
• Further, the video evaluation program of the present invention is characterized by causing a computer to function as an amount of change detection means for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in an input video signal and as an evaluation value calculation means for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture and outputting it to the outside.
• According to the invention described above, the amount of change is detected based on the plurality of frame pictures included in the input video signal and the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of change and the time interval between each frame picture. Therefore, it is possible to evaluate the smoothness in movement of a video at a frame rate during the period of video process in accordance with the amount of change of the input video signal input at a predetermined frame rate. Further, since the evaluation value obtained by the evaluation is output to the outside, it becomes possible to determine a frame rate based on the evaluation value. In other words, it is possible to determine a frame rate in accordance with the smoothness in movement of a video.
• In the video evaluation device of the present invention, it is preferable for the above-mentioned evaluation value calculation means to compute the amount of change in time in accordance with the amount of change and the time interval between each frame picture based on the amount of change and the time interval between each frame picture and calculate an evaluation value using the amount of change in time. With this, it is possible to calculate an evaluation value that takes into consideration the amount of change in time computed in accordance with the amount of change and the time interval calculated between each frame picture.
• In the video evaluation device of the present invention, it is preferable for the above-mentioned amount of change to be a value based on the difference in luminance value between each frame picture, or a movement vector between each frame picture. With this, it is possible to calculate an evaluation value based on the difference in luminance value between each frame picture or the movement vector between each frame picture.
• In the video evaluation device of the present invention, it is preferable to further comprise a feature value calculation means for calculating a movement feature value that indicates the feature of movement of an input video signal based on the above-mentioned amount of change, and for the above-mentioned evaluation value calculation means to calculate an evaluation value based on the movement feature value and the time interval between each frame picture. With this, it is possible to calculate an evaluation value that takes into consideration the movement feature value that indicates the feature of the movement of the input video signal.
• In the video evaluation device of the present invention, it is preferable for the above-mentioned evaluation value calculation means to compute the amount of change in time in accordance with the amount of change and the time interval between each frame picture based on the movement feature value and the time interval between each frame picture and calculate an evaluation value using the amount of change in time. With this, it is possible to calculate an evaluation value that takes into consideration the amount of change in time in accordance with the amount of change and the time interval between each frame picture.
• In the video evaluation device of the present invention, it is preferable for the above-mentioned amount of change to be a movement vector between each frame picture and for the above-mentioned movement feature value to be a value calculated based on the magnitude of the movement vector. With this, it is possible to calculate an evaluation value based on the magnitude of the movement vector between each frame.
• The frame rate determination device of the present invention is characterized by comprising a frame rate generation means for generating a first frame rate, an amount of change detection means for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in an input video signal, an evaluation value calculation means for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, and a frame rate determination means for determining a second frame rate for carrying out the process of an input video signal using the evaluation value and outputting it to the outside.
• The frame rate determination method of the present invention is characterized by comprising a frame rate generation step for generating a first frame rate, an amount of change detection step for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in an input video signal, an evaluation value calculation step for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, and a frame rate determination step for determining a second frame rate for carrying out the process of an input video signal using the evaluation value and outputting it to the outside.
• Further, the frame rate determination program of the present invention is characterized by causing a computer to function as a frame rate generation means for generating a first frame rate, an amount of change detection means for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in an input video signal, an evaluation value calculation means for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, and a frame rate determination means for determining a second frame rate for carrying out the process of an input video signal using the evaluation value and outputting it to the outside.
• According to the present invention described above, the amount of change is detected based on the plurality of frame pictures included in the input video signal, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. In other words, in accordance with the amount of change of the input video signal input according to the first frame rate, the smoothness in movement of the video at the first frame rate is evaluated and at the same time, the second frame rate is determined using the evaluation. Therefore, it is possible to read the input video signal at the second frame rate determined in accordance with the evaluation of the smoothness in movement of the video. In other words, it is possible to determine a frame rate of an input video signal in accordance with the smoothness in movement of a video and at the same time, to read the input video signal while maintaining the smoothness in movement of the video.
• In the frame rate determination device of the present invention, it is preferable for the above-mentioned frame rate determination means to set the second frame rate smaller than the first frame rate when the evaluation value is greater than a predetermined set value and to set the second frame rate greater than the first frame rate when the evaluation value is smaller than the predetermined set value by comparing the evaluation value with the predetermined set value. With this, it is possible to determine the second frame rate that is considered such that the evaluation value for evaluating the smoothness in movement of a video falls within a predetermined evaluation criterion range. In other words, it is possible to read the input video signal while keeping the smoothness in movement of a video within a predetermined criterion range.
• The video process device of the present invention is characterized by comprising a buffer means for storing an input video signal, a frame rate generation means for generating a first frame rate, an amount of change detection means for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in the input video signal, an evaluation value calculation means for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, a frame rate determination means for determining a second frame rate for carrying out the process of an input video signal using the evaluation value, and a video process means for reading the input video signal stored in the buffer means using the second frame rate and carrying out the video process.
• The video process method of the present invention is characterized by comprising a buffer step for storing an input video signal, a frame rate generation step for generating a first frame rate, an amount of change detection step for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in the input video signal, an evaluation value calculation step for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, a frame rate determination step for determining a second frame rate for carrying out the process of an input video signal using the evaluation value, and a video process step for reading the input video signal stored in the buffer means using the second frame rate and carrying out the video process.
• Further, the video process program of the present invention is characterized by causing a computer to function as a buffer means for storing an input video signal, a frame rate generation means for generating a first frame rate, an amount of change detection means for detecting an amount of change that indicates the degree of change between each frame picture based on a plurality of frame pictures included in the input video signal, an evaluation value calculation means for calculating an evaluation value relating to the smoothness in movement of an input video signal based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, a frame rate determination means for determining a second frame rate for carrying out the process of an input video signal using the evaluation value, and a video process means for reading the input video signal stored in the buffer means using the second frame rate and carrying out the video process.
• According to the present invention described above, the amount of change is detected based on the plurality of frame pictures included in the input video signal, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of change and the time interval between each frame picture corresponding to the first frame rate, and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. Further, the video process of the input video signal is carried out using the second frame rate. In other words, in accordance with the amount of change of the input video signal input according to the first frame rate, the smoothness in movement of the video at the first frame rate is evaluated and at the same time, the second frame rate is determined using the evaluation. Further, the video process of the input video signal is carried out using the second frame rate determined based on the evaluation value for the input video signal. Therefore, it is possible to carry out the video process of the input video signal based on the second frame rate determined in accordance with the evaluation of the smoothness in movement. In other words, it is possible to determine a frame rate in accordance with the smoothness in movement of a video and at the same time, to carry out the video process of an input video signal while maintaining the smoothness in movement of the video.
• [Other aspects] The video evaluation device of the present invention is characterized by comprising an amount of displacement detection means for detecting an amount of displacement in movement that indicates the degree of displacement of an input video signal based on a plurality of frame pictures included in the input video signal and an evaluation value calculation means for calculating and outputting an evaluation value for evaluating the smoothness in movement of the input video signal based on the amount of displacement in movement and the frame rate of the input video signal.
• The video evaluation method of the present invention is characterized by comprising an amount of displacement in movement detection step for detecting an amount of displacement in movement that indicates the degree of displacement of an input video signal based on a plurality of frame pictures included in the input video signal and an evaluation value calculation step for calculating and outputting an evaluation value for evaluating the smoothness in movement of the input video signal based on the amount of displacement in movement and the frame rate of the input video signal.
• Further, the video evaluation program of the present invention is characterized by causing a computer to function as an amount of displacement detection means for detecting an amount of displacement in movement that indicates the degree of displacement of an input video signal based on a plurality of frame pictures included in the input video signal and an evaluation value calculation means for calculating an evaluation value for evaluating the smoothness in movement of the input video signal based on the amount of displacement in movement and the frame rate of the input video signal and outputting it to the outside.
• According to the invention described above, the amount of displacement in movement is detected based on the plurality of frame pictures included in the input video signal and the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of displacement in movement and the frame rate of the input video signal. Therefore, it is possible to evaluate the smoothness in movement of a video at a frame rate during the period of video process in accordance with the amount of displacement in movement of the input video signal input at a predetermined frame rate. Further, since the evaluation value obtained by the evaluation is output to the outside, it becomes possible to determine a frame rate based on the evaluation value. In other words, it is possible to determine a frame rate in accordance with the smoothness in movement of a video.
• In the video evaluation device of the present invention, it is preferable to further comprise a feature value calculation means for calculating a movement feature value that indicates the feature of movement of an input video signal based on the above-mentioned amount of displacement in movement, and for the above-mentioned evaluation value calculation means to calculate an evaluation value based on the movement feature value and the frame rate of the input video signal. With this, it is possible to calculate an evaluation value that takes into consideration the movement feature value that indicates the feature of the movement of the input video signal.
• In the video evaluation device of the present invention, it is preferable for the above-mentioned evaluation value calculation means to compute an amount of movement that indicates the degree of movement between each frame based on the amount of displacement in movement and the frame rate of the input video signal and to calculate an evaluation value using the amount of movement. Further, it is preferable for the above-mentioned evaluation value calculation means to compute an amount of movement that indicates the degree of movement between each frame based on the movement feature value and the frame rate of the input video signal and to calculate an evaluation value using the amount of movement. With this, it is possible to calculate an evaluation value that takes into consideration the amount of movement that indicates the degree of movement between each frame.
• In the video evaluation device of the present invention, it is preferable for the above-mentioned amount of displacement in movement to be a movement vector and for the above-mentioned feature value to be a value calculated based on the magnitude of the movement vector. With this, it is possible to calculate an evaluation value based on the magnitude of the movement vector between each frame.
• The frame rate determination device of the present invention is characterized by comprising a frame rate generation means for generating a first frame rate, an amount of displacement detection means for detecting an amount of displacement in movement that indicates the degree of displacement of an input video signal, an evaluation value calculation means for calculating an evaluation value for evaluating the smoothness in movement of an input video signal based on the amount of displacement in movement and the first frame rate, and a frame rate determination means for determining a second frame rate for carrying out the process of an input video signal using the evaluation value and outputting it to the outside.
• According to the present invention, the amount of displacement in movement is detected based on the plurality of frame pictures included in the input video signal, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of displacement in movement and the first frame rate, and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. In other words, in accordance with the amount of displacement in movement of the input video signal input according to the first frame rate, the smoothness in movement of the video at the first frame rate is evaluated and at the same time, the second frame rate is determined using the evaluation. Therefore, it is possible to read the input video signal at the second frame rate determined in accordance with the evaluation of the smoothness in movement of the video. In other words, it is possible to determine a frame rate of an input video signal in accordance with the smoothness in movement of the video and at the same time, to read an input video signal while maintaining the smoothness in movement of the video.
• In the frame rate determination device of the present invention, it is preferable for the above-mentioned frame rate determination means to set the second frame rate smaller than the first frame rate when the evaluation value is greater than a predetermined set value and to set the second frame rate greater than the first frame rate when the evaluation value is smaller than the predetermined set value by comparing the evaluation value with the predetermined set value. With this, it is possible to determine the second frame rate that is considered such that the evaluation value for evaluating the smoothness in movement of a video falls within a predetermined evaluation criterion range. In other words, it is possible to read the input video signal while keeping the smoothness in movement of a video within a predetermined criterion range.
• The video process device of the present invention is characterized by comprising a buffer means for storing an input video signal, a frame rate generation means for generating a first frame rate, an amount of displacement detection means for detecting an amount of displacement in movement that indicates the degree of movement of an input video signal based on a plurality of frame pictures included in the input video signal, an evaluation value calculation means for calculating an evaluation value for evaluating the smoothness in movement of an input video signal based on the amount of displacement in movement and the first frame rate, a frame rate determination means for determining a second frame rate for carrying out the process of an input video signal using the evaluation value, and a video process means for reading the input video signal stored in the buffer means using the second frame rate and carrying out the video process.
• According to the present invention, the amount of displacement in movement is detected based on the plurality of frame pictures included in the input video signal, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of displacement in movement and the first frame rate, and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. Further, the video process of the input video signal is carried out using the second frame rate. In other words, in accordance with the amount of displacement in movement of the input video signal input according to the first frame rate, the smoothness in movement of the video at the first frame rate is evaluated and at the same time, the second frame rate is determined using the evaluation. Further, the video process of the input video signal is carried out using the second frame rate determined based on the evaluation value for the input video signal. Therefore, it is possible to carry out the video process of the input video signal based on the second frame rate determined in accordance with the evaluation of the smoothness in movement. In other words, it is possible to determine a frame rate in accordance with the smoothness in movement of a video and at the same time, to carry out the video process of an input video signal while maintaining the smoothness in movement of the video.
• According to the video evaluation device, the frame rate determination device, the video process device, the video evaluation method, the frame rate determination method, the video process method, the video evaluation program, the frame rate determination program, and the video process program of the present invention, since it is possible to determine a frame rate in accordance with the smoothness in movement of a video, it becomes possible to provide a video with smoothness in movement while reducing the amount of process, the amount of data, and the power consumption.
BRIEF DESCRIPTION OF THE DRAWINGS
• FIG. 1 is a diagram illustrating a functional configuration of video evaluation device in a modification example of a first embodiment.
• FIG. 2 is a diagram for explaining a method for detecting an amount of displacement. (a) shows a frame picture P0 and (b) is a diagram showing a frame picture P1.
• FIG. 3 is a diagram for explaining a method for evaluating the smoothness in movement. (a) is a diagram for explaining a method for computing an amount of movement between frames and (b) is a diagram for explaining a method for computing an evaluation value based on an amount of movement between frames.
• FIG. 4 is a flow chart showing a video evaluation process in a modification example of the first embodiment.
• FIG. 5 is a diagram for explaining a range obtained based on the magnitude and direction of each movement vector.
• FIG. 6 is a diagram illustrating a module configuration of a video evaluation program in a modification example of the first embodiment.
• FIG. 7 is a diagram illustrating a functional configuration of frame rate determination device in a second embodiment.
• FIG. 8 (a) illustrates a first frame rate and (b) is a diagram illustrating a sampling rate of an input video signal.
• FIG. 9 is a flow chart showing a flow of a frame rate determination process in the second embodiment.
• FIG. 10 is a diagram illustrating a module configuration of a frame rate determination program in the second embodiment.
• FIG. 11 is a diagram illustrating a functional configuration of video process device in a third embodiment.
• FIG. 12 is a flow chart showing a flow of a video process in the third embodiment.
• FIG. 13 is a diagram illustrating a module configuration of a video process program in the third embodiment.
• FIG. 14 is a diagram illustrating a functional configuration of video evaluation device in the first embodiment.
• FIG. 15 is a diagram for explaining a method for detecting an amount of change.
• FIG. 16 is a diagram for explaining a method for evaluating the smoothness in movement. (a) is a diagram for explaining a method for computing an amount of change in time and (b) is a diagram for explaining a method for calculating an evaluation value based on an amount of change in time.
• FIG. 17 is a flow chart showing a flow of a video evaluation process in the first embodiment.
• FIG. 18 is a diagram illustrating a module configuration of a video evaluation program in the first embodiment.
DESCRIPTION OF THE PREFERRED EMBODIMENTS
• Each embodiment of video evaluation device, frame rate determination device, video process device, a video evaluation method, a frame rate determination method, a video process method, a video evaluation program, a frame rate determination program, and a video process program according to the present invention is explained below based on drawings. The same symbols are attached to the same components in each drawing and no duplicated explanation will be given here.
First Embodiment
• First, a first embodiment of the present invention is explained below.FIG. 14 is a diagram illustrating a functional configuration ofvideo evaluation device140 in the first embodiment.
• Here, thevideo evaluation device140 is physically a computer comprising a CPU (Central Processing Unit), storage units such as memory, communication devices, etc. Therefore, thevideo evaluation device140 may be a fixed communication terminal such as PC terminal or may be a mobile communication terminal such as mobile phone. In other words, as thevideo evaluation device140, device capable of processing information can be applied widely.
• The functional configuration of thevideo evaluation device140 is explained with reference toFIG. 14. As shown inFIG. 14, thevideo evaluation device140 has an amount ofchange detection section1401 and an evaluationvalue calculation section1402.
• The amount ofchange detection section1401 divides aninput video signal1403 input as a moving picture signal from the outside into frame pictures. The amount ofchange detection section1401 detects an amount of change that indicates the degree of change between each frame picture of the input video signal based on the plurality of divided frame pictures. The amount ofchange detection section1401 outputs a detected amount ofchange1405 to the evaluationvalue calculation section1402.
• Here, a method for detecting the amount ofchange1405 is explained specifically with reference toFIG. 15. The amount ofchange detection section1401 sequentially reads two successive frame pictures divided from theinput video signal1403. Here, for simplicity of explanation, the two successive frame pictures are referred to as a frame picture P0 and a frame picture P1 in order of read. The amount ofchange detection section1401 finds the difference in luminance value between pixels located at the same coordinates in the read frame picture P1 and frame picture P0 and calculates the square of the difference for each pixel included in the entire frame picture. The amount ofchange detection section1401 detects the amount ofchange1405 by calculating the average value of the above-mentioned squared values calculated for each pixel. Therefore, the calculated average value is output to the evaluationvalue calculation section1402 as the amount ofchange1405.
• The evaluationvalue calculation section1402 computes an amount of change in time in accordance with the amount ofchange1405 and the time interval between each frame picture based on the amount ofchange1405 received from the amount ofchange detection section1401 and the time interval between each frame picture based onframe rate information1404 received from the outside. The evaluationvalue calculation section1402 calculates an evaluation value for evaluating the smoothness in movement of an input video based on the computed amount of change in time. The evaluationvalue calculation section1402 outputs a calculatedevaluation value1406 to the outside. Here, the outside corresponds, for example, to device such as one for determining an optimum frame rate for carrying out the vide process of theinput video signal1403 based on theevaluation value1406. By outputting theevaluation value1406 to such external device, it becomes possible to determine a frame rate in accordance with the smoothness in movement of the video of theinput video signal1403.
• Here, a method for evaluating the smoothness in movement is explained specifically with reference toFIG. 16. First, for example, a method for computing an amount of change in time S1 at a time T1 is explained with reference toFIG. 16 (a). The evaluationvalue calculation section1402 computes the amount of change in time S1 based on a time interval Δt1 between a time T0 of the frame picture P0 and the time T1 of the frame picture P1 based on theframe rate information1404 and an amount of change Δc1. The time interval Δt1 between the time T0 and the time T1 will be 1/F1 sec when the frame rate of theframe rate information1404 at the time T1 is, for example, F1 fps.
• The method for computing an amount of change in time is explained more specifically. As shown inFIG. 16 (a), the amount of change in time S1 at the time T1 will be Δt1·Δc1 (the area of the portion S1 hatched with slash lines shown inFIG. 16 (a)) when, for example, the time interval between the time T0 and the time T1 is assumed to be Δt1 and the amount of change at the time T1 is assumed to be Δc1. Similarly, an amount of change in time S2 at a time T2 will be Δt2·Δc2 (the area of the portion S2 hatched with slash lines shown inFIG. 16 (a)) when, for example, the time interval between the time T1 and the time T2 is assumed to be Δt2 and the amount of change at the time T2 is assumed to be Δc2.
• Next, a method for calculating an evaluation value based on an amount of change in time is explained with reference toFIG. 16 (b). The evaluationvalue calculation section1402 calculates an evaluation value of the smoothness in movement of each frame picture at a time Tn (n: positive integer, this also applies hereinafter) for all of the frame pictures included in an input video signal. This is explained specifically. The evaluationvalue calculation section1402 calculates an evaluation value of the smoothness in movement at the time Tn using an amount of change in time Sn of each frame picture at the time Tn. This is explained more specifically. The evaluationvalue calculation section1402 calculates an evaluation value of the smoothness in movement at the time Tn using, for example, an expression α/Sn (α is a constant) including the amount of change in time Sn. Further, it may also be possible for the evaluationvalue calculation section1402 to calculate an evaluation value of the smoothness in movement at the time Tn using an expression a×exp^−bSn+c (a, b, c are constants) including the amount of change in time Sn. The evaluationvalue calculation section1402 calculates the average value of the evaluation values of the smoothness in movement at all the times of an input video. The average value is output to the outside as thefinal evaluation value1406 of the input video.
• Next, a flow of a video evaluation process in thevideo evaluation device140 in the first embodiment is explained below with reference toFIG. 17.
• First, the amount ofchange detection section1401 sequentially reads the two successive frame pictures (the frame picture P0, the frame picture P1) divided from the input video signal1403 (step S1701).
• Next, the amount ofchange detection section1401 finds the difference in luminance value between pixels located at the same coordinates in the read frame picture P1 and frame picture P0 and calculates the square of the difference for each pixel included in the entire frame picture (step S1702).
• Next, the amount ofchange detection section1401 calculates the average value of the above-mentioned squared values calculated for each pixel (step S1703). The calculated average value is output to the evaluationvalue calculation section1402 as the amount ofchange1405.
• Next, the evaluationvalue calculation section1402 calculates an amount of change in time in accordance with the amount ofchange1405 and the time interval between each frame picture based on the amount ofchange1405 received from the amount ofchange detection section1401 and the time interval between each frame picture based on theframe rate information1404 received from the outside (step S1704).
• Next, the evaluationvalue calculation section1402 calculates an evaluation value of the smoothness in movement at the time Tn of each frame picture for all the frame pictures included in the input video signal using the amount of change in time (step S1705).
• Next, the evaluationvalue calculation section1402 calculates the average value of the evaluation values of the smoothness in movement at all the times of the input video based on each of the calculated evaluation values (step S1706).
• Next, the evaluationvalue calculation section1402 outputs the calculated average value to the outside as theevaluation value1406 of the smoothness in movement for the entire input video signal1403 (step S1707).
• As described above, according to thevideo evaluation device140 in the first embodiment, the amount of change is calculated based on the squared value of the difference in luminance value between each frame of the plurality of frame pictures included in the input video signal. Further, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of change and the time interval between frame pictures based on the frame rate of the input video signal. Therefore, it is possible to evaluate the smoothness in movement of the video at the frame rate during the period of video process based on the amount of change in luminance value of the input video signal input in accordance with the predetermined frame rate. Further, it becomes possible to determine a frame rate outside based on the evaluation value because the evaluation value obtained by the evaluation is output to the outside. In other words, it is possible to determine a frame rate in accordance with the smoothness in movement of the video.
• The unit used for the amount of change detection by the amount ofchange detection section1401 described above is not limited to the above-mentioned entire frame picture. For example, the unit may be a block, a pixel, an object region, etc.
• The method of the amount of change detection by the amount ofchange detection section1401 is not limited to that described above, in which the average value of the squared values calculated for each pixel included in the frame picture is used. For example, it may also be possible to use the maximum value, the median value, or the minimum value of the above-mentioned squared values calculated for each pixel, or to use the square root of the maximum value, the median value, or the minimum value of the above-mentioned squared values, or to use the variance of the above-mentioned squared values for the entire frame picture.
• The method of the amount of change detection by the amount ofchange detection section1401 is not limited to that described above, in which the squared value of the difference in luminance value between pixels located at the same coordinates in each frame is used. For example, it may also be possible to use the value of difference in luminance value between pixels located at the same coordinates in each frame, or the absolute value of the difference.
• In the method of the amount of change detection by the amount ofchange detection section1401, it is possible to use any value that indicates the change in the input video signal between frames such as the movement vector between frame pictures of the input video signal, in addition to those values described above.
• The method for computing the amount of change in time by the evaluationvalue calculation section1402 is not limited to that described above, in which the above-mentioned expression (for example, Δt1·Δc1) is used, but it is only necessary for the method to be capable of computing it based on the amount of change and the time interval between frame pictures based on the frame rate information.
• The amount of change in time computed by the evaluationvalue calculation section1402 is not necessarily one for each frame. For example, it may be one for each block, pixel, or object.
• Thefinal evaluation value1406 of the input video calculated by the evaluationvalue calculation section1402 is not limited to the above-mentioned average value of the evaluation values of the smoothness in movement at all the times of the input video. For example, it may be the maximum value, the median value, or the minimum value of the evaluation values of the smoothness in movement at all the times of the input video.
• Thefinal evaluation value1406 of the input video calculated by the evaluationvalue calculation section1402 is not necessarily one for all of the frame pictures included in the input video signal. For example, it may be one for some frame pictures, or for each frame picture, block, pixel, or object.
• Finally, avideo evaluation program180 for causing a computer to function as the above-mentionedvideo evaluation device140 is explained with reference toFIG. 18.
• As shown inFIG. 18, thevideo evaluation program180 comprises amain module program1801 for generalizing processes, an amount ofchange detection module1802, and an evaluationvalue calculation module1803. The functions that the amount ofchange detection module1802 and the evaluationvalue calculation module1803 cause a computer to carry out are the same as those possessed by the above-mentioned amount ofchange detection section1401 and the evaluationvalue calculation section1402.
• Thevideo evaluation program180 is provided by, for example, storage media such as CD-ROM, DVD, and ROM or semiconductor memories. It may also be possible for thevideo evaluation program180 to be provided via a network as a computer data signal multiplexed on carriers.
Modification Example of the First Embodiment
• Next, a modification example of the above-mentioned first embodiment is explained.FIG. 1 is a diagram illustrating a functional configuration ofvideo evaluation device10 in a modification example of the first embodiment.
• Here, thevideo evaluation device10 is physically a computer comprising a CPU (Central Processing Unit), storage units such as memory, communication devices, etc. Therefore, thevideo evaluation device10 may be a fixed communication terminal such as PC terminal or may be a mobile communication terminal such as mobile phone. In other words, as thevideo evaluation device10, device capable of processing information can be applied widely.
• The functional configuration of thevideo evaluation device10 is explained with reference toFIG. 1. As shown inFIG. 1, thevideo evaluation device10 has an amount ofdisplacement detection section101, a featurevalue calculation section102, and an evaluationvalue calculation section103.
• The amount ofdisplacement detection section101 divides aninput video signal104 input as a moving picture signal from the outside into frame pictures. The amount ofdisplacement detection section101 detects an amount of displacement (an amount of displacement in movement) that indicates the degree of displacement in movement of the input video signal based on the plurality of divided frame pictures. The amount ofdisplacement detection section101 outputs a detected amount ofdisplacement106 to the featurevalue calculation section102.
• The amount of displacement is not limited to that which indicates the degree of displacement in movement of an input video signal, but it is only necessary for the amount of displacement to indicate the degree of change between each frame picture of an input video signal.
• Here, a method for detecting the amount ofdisplacement106 is explained specifically with reference toFIG. 2. The amount ofdisplacement detection section101 sequentially reads two successive frame pictures divided from theinput video signal104. Here, for simplicity of explanation, the two successive frame pictures are referred to as the frame picture P0 (refer toFIG. 2 (a)) and the frame picture P1 (refer toFIG. 2 (b)) in order of read. The amount ofdisplacement detection section101 divides the read frame picture P1 into blocks having a predetermined size. The amount ofdisplacement detection section101 searches the frame picture P0 for a picture signal pattern that best resembles the picture signal pattern of each block for each block of the frame picture P1. This search can be realized by using, for example, a search process by the block matching (correlation method) shown inFIG. 2. The amount ofdisplacement detection section101 detects a movement vector V (MVx, MVy) that is a spatial amount of displacement between signal patterns of both pictures based on the signal patterns of both pictures, judged to resemble each other by the search process. The movement vector V is output to the featurevalue calculation section102 as the amount ofdisplacement106.
• The featurevalue calculation section102 calculates amovement feature value107 that indicates the movement feature of an input video signal based on the amount ofdisplacement106 received from the amount of displacement detection section101. The featurevalue calculation section102 outputs the calculatedmovement feature value107 to the evaluationvalue calculation section103.
• Here, a method for calculating a movement feature value is explained specifically. The featurevalue calculation section102 finds the magnitude of the movement vector of each block of the frame picture P1 using the movement vector of each block of the frame picture P1 received as the amount ofdisplacement106. If, for example, it is assumed that the x component and the y component of the movement vector of an arbitrary block on the frame picture P1 are referred to as MVx and MVy, respectively, the magnitude of the movement vector can be obtained from (MVx²+MVy²)^1/2. The featurevalue calculation section102 calculates a value that features the frame picture P1 based on the magnitude of each movement vector. The calculated value is output to the evaluationvalue calculation section103 as themovement feature value107. As the value (the movement feature value107) calculated by the featurevalue calculation section102, for example, the maximum value, the median value, or the minimum value of the magnitudes of the movement vectors obtained for each block included in the frame picture apply.
• The evaluationvalue calculation section103 computes an amount of movement between frames that indicates the degree of movement between each frame picture based on themovement feature value107 received from the featurevalue calculation section102 and the time interval between frame pictures based onframe rate information105 received from the outside. The evaluationvalue calculation section103 calculates an evaluation value for evaluating the smoothness in movement of an input video signal based on the computed amount of movement between frames. The evaluationvalue calculation section103 outputs a calculatedevaluation value108 to the outside. Here, as the outside, for example, device etc. applies, which determines an optimum frame rate for carrying out the video process of theinput video signal104 based on theevaluation value108. By outputting theevaluation value108 to such external device, it becomes possible to determine a frame rate in accordance with the smoothness in movement of the video of theinput video signal104.
• When calculating an evaluation value, it is not necessarily required to calculate it based the amount of movement between frames. For example, it may also be possible to calculate an evaluation value based on the amount ofdisplacement106 and the amount of change in time in accordance with the time interval between each frame picture. It is possible to compute the amount of change in time based on themovement feature value107 received from the featurevalue calculation section102 and the time interval between frame pictures based on theframe rate information105 received from the outside.
• Here, a method for evaluating the smoothness in movement is explained specifically with reference toFIG. 3. First, for example, a method for computing the amount of movement between frames S1 at the time T1 is explained with reference toFIG. 3 (a). The evaluationvalue calculation section103 computes the amount of movement between frames S1 based on the time interval Δt1 between the time T0 of the frame picture P0 and the time T1 of the frame picture P1 based on theframe rate information105 and a movement feature value Δd1 of the frame picture P1. The time interval Δt1 between the time T0 and the time T1 will be 1/F1 sec when the frame rate of theframe rate information105 at the time T1 is, for example, F1 fps.
• The method for computing an amount of movement between frames is explained more specifically. As shown inFIG. 3 (a), the amount of movement between frames S1 at the time T1 will be Δt1·Δd1/2 (the area of the portion S1 hatched with slash lines shown inFIG. 3 (a)) when, for example, the time interval between the time T0 and the time T1 is assumed to be Δt1 and the movement feature value at the time T1 is assumed to be Δd1. Similarly, an amount of movement between frames S2 at the time T2 will be Δt2·Δd2/2 (the area of the portion S2 hatched with slash lines shown inFIG. 3 (a)) when, for example, the time interval between the time T1 and the time T2 is assumed to be Δt2 and a movement feature value at the time T2 is assumed to be Δd2.
• Next, a method for calculating an evaluation value based on an amount of movement between frames is explained with reference toFIG. 3 (b). The evaluationvalue calculation section103 calculates an evaluation value of the smoothness in movement of each frame picture at the time Tn (n: positive integer, this also applies hereinafter) for all of the frame pictures included in an input video signal. This is explained specifically. The evaluationvalue calculation section103 calculates an evaluation value of the smoothness in movement at the time Tn using the amount of movement between frames Sn of each frame picture at the time Tn. This is explained more specifically. The evaluationvalue calculation section103 calculates an evaluation value of the smoothness in movement at the time Tn using, for example, the expression α/Sn (α is a constant) including the amount of movement between frames Sn. Further, it may also be possible for the evaluationvalue calculation section103 to calculate an evaluation value of the smoothness in movement at the time Tn using the expression a×exp^−bSn+c (a, b, c are constants) including the amount of movement between frames Sn. The evaluationvalue calculation section103 calculates the average value of the evaluation values of the smoothness in movement at all the times of an input video. The average value is output to the outside as thefinal evaluation value108 of the input video.
• Next, a flow of a video evaluation process in thevideo evaluation device10 in the modification example of the first embodiment is explained below with reference toFIG. 4.
• First, the amount ofdisplacement detection section101 sequentially reads the two successive frame pictures (the frame picture P0, the frame picture P1) divided from the input video signal104 (step S401).
• Next, the amount ofdisplacement detection section101 divides the read frame picture P1 into blocks having a predetermined size (step S402).
• Next, the amount ofdisplacement detection section101 searches the frame picture P0 for a picture signal pattern that best resembles the picture signal pattern of each block for each block of the frame picture P1 (step S403).
• The amount ofdisplacement detection section101 detects a movement vector (MVx, MVy) that is a spatial amount of displacement between the picture signal patterns based on the signal patterns of both pictures judged to resemble each other by the search (step S404). The detected movement vector is output to the featurevalue calculation section102 as the amount ofdisplacement106.
• Next, the featurevalue calculation section102 finds the magnitude of the movement vector of each block of the frame picture P1 using the movement vector of each block of the frame picture P1 included in the amount ofdisplacement106.
• Next, the featurevalue calculation section102 calculates a value that features the frame picture P1 based on the magnitude of each movement vector (step S406). The calculated value is output to the evaluationvalue calculation section103 as themovement feature value107.
• Next, the evaluationvalue calculation section103 calculates an amount of displacement between frames, which is an amount of movement between each frame picture, based on themovement feature value107 received from the featurevalue calculation section102 and the time interval between each frame picture based on theframe rate information105 received from the outside (step S407).
• Next, the evaluationvalue calculation section103 calculates an evaluation value of the smoothness in movement at the time Tn of each frame picture for all the frame pictures included in the input video signal using the amount of movement between frames (step S408).
• Next, the evaluationvalue calculation section103 calculates the average value of the evaluation values of the smoothness in movement at all the times of the input video based on each of the calculated evaluation values (step S409).
• Next, the evaluationvalue calculation section103 outputs the calculated average value to the outside as theevaluation value108 of the smoothness in movement for the entire input video signal104 (step S410).
• As described above, according to thevideo evaluation device10 in the modification example of the first embodiment, the movement vector is detected based on the plurality of frame pictures included in the input video signal and the movement feature value is calculated based on the magnitude of the movement vector. Further, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the movement feature value and the time interval between frame pictures based on the frame rate of the input video signal. Therefore, it is possible to evaluate the smoothness in movement of the video at the frame rate during the period of video process based on the magnitude of the movement vector of the input video signal input in accordance with the predetermined frame rate. Further, it becomes possible to determine a frame rate outside based on the evaluation value because the evaluation value obtained by the evaluation is output to the outside. In other words, it is possible to determine a frame rate in accordance with the smoothness in movement of the video.
• The unit used for searching for a picture signal pattern by the above-mentioned amount ofdisplacement detection section101 is not limited to the block. For example, the unit may be a frame, a pixel, an object region, etc. Further, the method of the search process by the amount ofdisplacement detection section101 is not limited to the above-mentioned block matching. For example, it may be a concentration gradient method.
• The amount ofdisplacement detection section101 may read theinput video signal104 including the movement vector of a video as the above-mentionedinput video signal104. In this case, the amount ofdisplacement detection section101 detects the movement vector from theinput video signal104 received from the outside and outputs the detected movement vector to the featurevalue calculation section102 as the amount ofdisplacement106.
• Themovement feature value107 calculated by the featurevalue calculation section102 is not limited to the maximum value, the average value, the median value, or the minimum value of the magnitudes of the movement vectors obtained for each of the blocks as described above. For example, it may be the magnitude of one movement vector obtained for each frame picture or the maximum value, the average value, the median value, or the minimum value of the magnitudes of the movement vectors obtained for each pixel in the frame picture or for each object region.
• Themovement feature value107 calculated by the featurevalue calculation section102 is not necessarily required to be one for each frame. For example, it may be one for each block, pixel, or object. Further, themovement feature value107 may be one for each of ranges R (for example, R1, R2, R3) defined by the plurality of circles with the origin shown inFIG. 5 being their center and the plurality of lines extending from the origin with respect to the distribution of the movement vectors obtained for each block, pixel, or object.
• Here, a method for calculating themovement feature value107 for each of the above-mentioned ranges R is explained specifically with reference toFIG. 5, a case where one frame picture is divided into nine blocks being an example. First, a movement vector is obtained for each of the nine divided blocks. It is assumed that each of the obtained movement vectors is referred to as V1 to V9. Next, each of the movement vectors V1 to V9 is projected on a graph shown inFIG. 5. For example, it is assumed that the movement vectors V1 to V4 are included in the range R1 shown inFIG. 5, the movement vectors V5 and V6 are included in the range R2 shown inFIG. 5, and the movement vectors V7 to V9 are included in the range R3 shown inFIG. 5. In this case, for example, a movement vector VR1 calculated as the average value of the movement vectors V1 to V4 is obtained as themovement feature value107 in the range R1, a movement vector VR2 calculated as the average value of the movement vectors V5 and V6 is obtained as themovement feature value107 in the range R2, and a movement vector VR3 calculated as the average value of the movement vectors V7 to V9 is obtained as themovement feature value107 in the range R3, as a result. The method for obtaining a movement vector is not limited to the method that uses a block as the unit, and the unit may be, for example, a pixel or an object.
• The method for computing the amount of movement between frames by the evaluationvalue calculation section103 is not limited to the method that uses the above-mentioned expression (for example, Δt1·Δd1/2). For example, it may be computed from the expression β·MVx·Δt1/2+γ·MVy·Δt1/2 (β, γ are constants) expressed by using the movement vector (MVx, MVy), which is the amount ofdisplacement106, and time interval between frame pictures based on theframe rate information105.
• The amount of movement between frames computed by the evaluationvalue calculation section103 is not necessarily required to be one for each frame. For example, it may be one for each block, pixel, object, or for each of the above-mentioned ranges R (refer toFIG. 5).
• The calculation of the evaluation value of the smoothness in movement at the time Tn of each frame picture calculated by the evaluationvalue calculation section103 is not limited to the calculation using the above-mentioned α/Sn (α is a constant) or a×exp^−bSn+c (a, b, c are constants). For example, it may be calculated from a function using the movement vector (MVx, MVy), which is the amount ofdisplacement106, and the time interval between frame pictures based on theframe rate information105.
• Thefinal evaluation value108 of an input video signal calculated by the evaluationvalue calculation section103 is not limited to the above-mentioned average value of the evaluation values of the smoothness in movement at all the times of the input video. For example, it may be the maximum value, the median value, or the minimum value of the evaluation values of the smoothness in movement at all the times of the input video.
• Thefinal evaluation value108 of an input video calculated by the evaluationvalue calculation section103 is not necessarily required to be one for the entire frame picture included in the input video signal. For example, it may be one for some frame pictures, or for each frame picture, block, pixel, object, or for each of the above-mentioned ranges R (refer toFIG. 5).
• Finally, avideo evaluation program50 for causing a computer to function as the above-mentionedvideo evaluation device10 is explained with reference toFIG. 6.
• As shown inFIG. 6, thevideo evaluation program50 comprises amain module program501 for generalizing processes, an amount ofdisplacement detection module502, a featurevalue calculation module503, and an evaluationvalue calculation module504. The functions that the amount ofdisplacement detection module502, the featurevalue calculation module503, and the evaluationvalue calculation module504 cause a computer to carry out are the same as those possessed by the above-mentioned amount ofdisplacement detection section101, the featurevalue calculation section102, and the evaluationvalue calculation section103.
• Thevideo evaluation program50 is provided by, for example, storage media such as CD-ROM, DVD, and ROM or semiconductor memories. It may also be possible for thevideo evaluation program50 to be provided via a network as a computer data signal multiplexed on carriers.
• Further, it is possible to make thevideo evaluation device10 in the modification example of the first embodiment have the same functional configuration as that of thevideo evaluation device140 in the first embodiment by integrating the amount ofdisplacement detection section101 and the featurevalue calculation section102 into an amount of change detection section.
Second Embodiment
• Next, a second embodiment of the present invention is explained.FIG. 7 is a diagram illustrating a functional configuration of framerate determination device70 in the second embodiment.
• Here, the framerate determination device70 is physically a computer comprising a CPU (Central Processing Unit), storage units such as memory, communication devices, etc. Therefore, the framerate determination device70 may be a fixed communication terminal such as PC terminal or may be a mobile communication terminal such as mobile phone. In other words, as the framerate determination device70, device capable of processing information can be applied widely.
• The functional configuration of the framerate determination device70 is explained with reference toFIG. 7. As shown inFIG. 7, the framerate determination device70 comprises a framerate generation section701, avideo evaluation section702, and a framerate determination section703.
• The framerate generation section701 generates afirst frame rate705. The framerate generation section701 outputs the generatedfirst frame rate705 to thevideo evaluation section702 and the framerate determination section703.
• Thevideo evaluation section702 has the same function as that possessed by thevideo evaluation device140 described in the above-mentioned first embodiment or that possessed by thevideo evaluation device10 described in the modification example of the first embodiment. In other words, thevideo evaluation section702 has the same function as that possessed by the above-mentioned amount ofchange detection section1401 and the evaluationvalue calculation section1402 or that possessed by the amount ofdisplacement detection section101, the featurevalue calculation section102, and the evaluationvalue calculation section103.
• Thevideo evaluation section702 reads frame pictures at the time intervals in accordance with thefirst frame rate705 from aninput video signal704 input as a moving picture signal from the outside and calculates anevaluation value706 of the smoothness in movement for the entireinput video signal704. Thevideo evaluation section702 outputs the calculatedevaluation value706 of the smoothness in movement to the framerate determination section703.
• Here, there is no problem with thefirst frame rate705 even if it differs from the sampling rate of theinput video signal704. For example, while thefirst frame rate705 shown inFIG. 8 (a) is 1/15 sec, the sampling rate of theinput video signal704 shown inFIG. 8 (b) is 1/30 sec.
• The framerate determination section703 determines asecond frame rate707 based on theevaluation value706 received from thevideo evaluation section702 and thefirst frame rate705 received from the framerate generation section701. The framerate determination section703 outputs the determinedsecond frame rate707 to the framerate determination device70 as a frame rate for carrying out the process of theinput video signal704.
• This is explained specifically. When, for example, theevaluation value706 is greater than a predetermined set value, the framerate determination section703 decreases thesecond frame rate707 lower than thefirst frame rate705. When, for example, theevaluation value706 is less than the predetermined set value, the framerate determination section703 increases thesecond frame rate707 higher than thefirst frame rate705. When, for example, theevaluation value706 is equal to the predetermined set value, the framerate determination device70 makes thesecond frame rate707 equal to thefirst frame rate705.
• With this, it is possible to determine a second frame rate such that the evaluation value for evaluating the smoothness in movement of a video falls within a predetermined evaluation criterion range. In other words, it is possible to read an input video signal while keeping the smoothness in movement of a video within a predetermined criterion range. The above-mentioned predetermined set value may be set in advance or given from the outside.
• Next, the flow of a frame rate determination process in the framerate determination device70 of the second embodiment is explained with reference toFIG. 9.
• First, the framerate generation section701 generates the first frame rate705 (step S901).
• Next, thevideo evaluation section702 reads frame pictures at the time intervals in accordance with thefirst frame rate705 from the input video signal704 (step S902).
• Next, thevideo evaluation section702 calculates theevaluation value706 of the smoothness in movement for the entireinput video signal704 based on each frame picture (step S903). In other words, thevideo evaluation section702 calculates theevaluation value706 of the smoothness in movement for the entireinput video signal704 by carrying out, based on each frame picture, the video evaluation processes from step S1701 to step S1707 explained in the above-mentioned first embodiment (refer toFIG. 17) or the video evaluation processes from step S401 to step S410 explained in the above-mentioned modification example of the first embodiment (refer toFIG. 4).
• Next, the framerate determination section703 determines thesecond frame rate707 based on theevaluation value706 received from thevideo evaluation section702 and thefirst frame rate705 received from the frame rate generation section701 (step S904).
• Next, the framerate determination section703 outputs thesecond frame rate707 to the outside as a frame rate for carrying out the process of the input video signal704 (step S905).
• As described above, according to the framerate determination device70 of the second embodiment, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of change in the input video signal and the time interval between frame pictures based on the first frame rate and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. In other words, the smoothness in movement of the video at the first frame rate is evaluated in accordance with the amount of change in the input video signal input according to the first frame rate and at the same time, the second frame rate is determined using the evaluation.
• Further, the movement vector is detected based on the plurality of frame pictures included in the input video signal and the movement feature value is calculated based the magnitude of the movement vector. Then, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the movement feature value and the time interval between frame pictures based on the first frame rate and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. In other words, the smoothness in movement of the video at the first frame rate is evaluated in accordance with the movement feature value of the input video signal input according to the first frame rate and at the same time, the second frame rate is determined using the evaluation.
• Therefore, it is possible to read the input video signal while maintaining the smoothness in movement of a video as well as determining a frame rate of the input video signal in accordance with the smoothness in movement of the video.
• Finally, a framerate determination program100 for causing a computer to function as the above-mentioned framerate determination device70 is explained with reference toFIG. 10.
• As shown inFIG. 10, the framerate determination program100 comprises amain module program1001 for generalizing processes, a framerate generation module1002, avideo evaluation module1003, and a framerate determination module1004. The functions that the framerate generation module1002, thevideo evaluation module1003, and the framerate determination module1004 cause a computer to carry out are the same as those possessed by the above-mentioned framerate generation section701, thevideo evaluation section702, and the framerate determination section703.
• The framerate determination program100 is provided by, for example, storage media such as CD-ROM, DVD, and ROM or semiconductor memories. It may also be possible for the framerate determination program100 to be provided via a network as a computer data signal multiplexed on carriers.
Third Embodiment
• Next, a third embodiment is explained.FIG. 11 is a diagram illustrating a functional configuration ofvideo process device110 in the third embodiment.
• Here, thevideo process device110 is physically a computer comprising a CPU (Central Processing Unit), storage units such as memory, communication devices, etc. Therefore, thevideo process device110 may be a fixed communication terminal such as PC terminal or may be a mobile communication terminal such as mobile phone. In other words, as thevideo process device110, device capable of processing information can be applied widely.
• The functional configuration of thevideo process device110 is explained with reference toFIG. 11. As shown inFIG. 11, thevideo process device110 has abuffer section1101, a framerate determination section1102, and avideo process section1103.
• Thebuffer section1101 temporarily stores aninput video signal1104 input from the outside as a moving picture signal in the buffer on the memory. Thebuffer section1101 outputs theinput video signal1104 to the framerate determination section1102. Theinput video signal1104 temporarily stored in the buffer is read as aprocess video signal1106 by thevideo process section1103, which will be described later.
• The framerate determination section1102 has the same function as that possessed by the framerate determination device70 described in the above-mentioned second embodiment. In other words, the framerate determination section1102 has the same function as that possessed by the above-mentioned framerate generation section701, thevideo evaluation section702, and the framerate determination section703. The framerate determination section1102 calculates a videoprocess frame rate1105 corresponding to the above-mentionedsecond frame rate707 using theinput video signal1104 received from thebuffer section1101. The framerate determination section1102 outputs the calculated videoprocess frame rate1105 to thevideo process section1103.
• Thevideo process section1103 reads theprocess video signal1106 from thebuffer section1101 at the time intervals in accordance with the videoprocess frame rate1105 and carries out the video process based on theprocess video signal1106. This is explained specifically. Thevideo process section1103 reads only the frame pictures at the times corresponding to the videoprocess frame rate1105 received from the framerate determination section1102 among the frame pictures included in theprocess video signal1106 stored in thebuffer section1101 and carries out the video process. Thevideo process section1103 outputs the video generated by the video process to the outside. To the video process carried out by thevideo process section1103, for example, acquisition, storing, transmission, encoding, and decoding of videos correspond.
• Next, the flow of a video process in thevideo process device110 in the third embodiment is explained with reference toFIG. 12.
• First, thebuffer section1101 outputs theinput video signal1104 input from the outside as a moving picture signal to the framerate determination section1102 and at the same time, temporarily stores theinput video signal1104 in the buffer on the memory (step S1201).
• Next, the framerate determination section1102 calculates the videoprocess frame rate1105 using theinput video signal1104 received from the buffer section1101 (step S1202). In other words, the framerate determination section1102 calculates the videoprocess frame rate1105, which is a second frame rate, by carrying out the frame rate determination processes from step S901 to step S905 explained in the above-mentioned second embodiment using the input video signal1104 (refer toFIG. 9).
• Next, thevideo process section1103 reads theprocess video signal1106 from thebuffer section1101 at the time intervals in accordance with the videoprocess frame rate1105 and carries out the video process based on the process video signal1106 (step S1203).
• Next, thevideo process section1103 outputs the video generated by the video process to the outside (step S1204).
• As described above, according to thevideo process device110 in the third embodiment, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the amount of change in the input video signal and the time interval between frame pictures based on the first frame rate and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. Then, the video process of the input video signal is carried out using the second frame rate. In other words, the smoothness in movement of the video at the first frame rate is evaluated in accordance with the amount of change in the input video signal input according to the first frame rate and at the same time, the second frame rate is determined using the evaluation. Then, the video process of the input video signal is carried out using the second frame rate determined based on the evaluation value for the input video signal.
• Further, the movement vector is detected based on the plurality of frame pictures included in the input video signal and the movement feature value is calculated based on the magnitude of the movement vector. Then, the evaluation value for evaluating the smoothness in movement of the input video signal is calculated based on the movement feature value and the time interval between frame pictures based on the first frame rate and the second frame rate for carrying out the process of the input video signal is determined using the evaluation value. Further, the video process of the input video signal is carried out using the second frame rate. In other words, the smoothness in movement of the video at the first frame rate is evaluated in accordance with the movement feature value of the input video signal input according to the first frame rate and at the same time, the second frame rate is determined using the evaluation. Then, the video process of the input video signal is carried out using the second frame rate determined based on the evaluation value for the input video signal.
• Therefore, it is possible to carry out the video process of the input video signal based on the second frame rate determined in accordance with the evaluation of the smoothness in movement. In other words, it is possible to carry out the video process of the input video signal while maintaining the smoothness in movement of the video as well as determining a frame rate in accordance with the smoothness in movement of the video.
• The above-mentionedvideo process device110 can be applied to, for example, video acquisition device (for example, camera), video transmission device, a video encoder, or a video decoder.
• When thevideo process device110 is applied to video acquisition device, each of the functions described above functions as follows. Thebuffer section1101 buffers theinput video signal1104 input at a sampling rate (for example, 30 fps). The framerate determination section1102 calculates the optimum video process frame rate1105 (for example, 15 fps) when the video acquisition device acquires a video using theinput video signal1104 received from thebuffer section1101 and outputs it to thevideo process section1103. Thevideo process section1103 samples theinput video signal1104 stored in thebuffer section1101 using the video process frame rate1105 (for example, 15 fps) received from the framerate determination section1102. Thevideo process section1103 acquires theprocess video signal1106 obtained by sampling at the video process frame rate1105 (for example, 15 fps).
• When thevideo process device110 is applied to video storing device, each of the functions described above functions as follows. Thebuffer section1101 buffers theinput video signal1104 input at a sampling rate (for example, 30 fps). The framerate determination section1102 calculates the optimum video process frame rate1105 (for example, 15 fps) when the video storing device stores a video using theinput video signal1104 received from thebuffer section1101 and outputs it to thevideo process section1103. Thevideo process section1103 samples theinput video signal1104 stored in thebuffer section1101 using the video process frame rate1105 (for example, 15 fps) received from the framerate determination section1102. Thevideo process section1103 stores theprocess video signal1106 obtained by sampling at the video process frame rate1105 (for example, 15 fps).
• When thevideo process device110 is applied to video transmission device, each of the functions described above functions as follows. Thebuffer section1101 buffers theinput video signal1104 input at a sampling rate (for example, 30 fps). The framerate determination section1102 calculates the optimum video process frame rate1105 (for example, 15 fps) when the video transmission device transmits a video using theinput video signal1104 received from thebuffer section1101 and outputs it to thevideo process section1103. Thevideo process section1103 samples theinput video signal1104 stored in thebuffer section1101 using the video process frame rate1105 (for example, 15 fps) received from the framerate determination section1102. Thevideo process section1103 transmits theprocess video signal1106 obtained by sampling at the video process frame rate1105 (for example, 15 fps).
• When thevideo process device110 is applied to a video encoder, each of the functions described above functions as follows. Thebuffer section1101 buffers theinput video signal1104 input at a sampling rate (for example, 30 fps). The framerate determination section1102 calculates the optimum video process frame rate1105 (for example, 15 fps) when the video encoder encodes a video using theinput video signal1104 received from thebuffer section1101 and outputs it to thevideo process section1103. Thevideo process section1103 samples theinput video signal1104 stored in thebuffer section1101 using the video process frame rate1105 (for example, 15 fps) received from the framerate determination section1102. Thevideo process section1103 encodes theprocess video signal1106 obtained by sampling at the video process frame rate1105 (for example, 15 fps).
• When thevideo process device110 is applied to a video decoder, each of the functions described above functions as follows. Thebuffer section1101 buffers theinput video signal1104 input at a sampling rate (for example, 30 fps). The framerate determination section1102 calculates the optimum video process frame rate1105 (for example, 15 fps) when the video decoder decodes a video using theinput video signal1104 received from thebuffer section1101 and outputs it to thevideo process section1103. Thevideo process section1103 samples theinput video signal1104 stored in thebuffer section1101 using the video process frame rate1105 (for example, 15 fps) received from the framerate determination section1102. Thevideo process section1103 decodes theprocess video signal1106 obtained by sampling at the video process frame rate1105 (for example, 15 fps).
• Finally, avideo process program130 for causing a computer to function as the above-mentionedvideo process device110 is explained with reference toFIG. 13.
• As shown inFIG. 13, thevideo process program130 comprises amain module program1301 for generalizing processes, abuffer module1302, a framerate determination module1303, and avideo process module1304. The functions that thebuffer module1302, the framerate determination module1303, and thevideo process module1304 cause a computer to carry out are the same as those possessed by the above-mentionedbuffer section1101, the framerate determination section1102, and thevideo process section1103.
• Thevideo process program130 is provided by, for example, storage media such as CD-ROM, DVD, and ROM or semiconductor memories. It may also be possible for thevideo process program130 to be provided via a network as a computer data signal multiplexed on carriers.

Claims (15)

1. A video evaluation device comprising:

an amount of change detection section configured to detect, based on a plurality of frame pictures included in an input video signal, an amount of change that indicates the degree of change between each of the frame pictures; and

an evaluation value calculation section configured to calculate an evaluation value relating to the smoothness in movement of the input video signal based on the amount of change and the time interval between each of the frame pictures and outputting the evaluation value to the outside, wherein

the evaluation value calculation section computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the amount of change and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time,

the amount of change is a value based on a difference in luminance value between pixels located at the same coordinates in the successive frame pictures, and

the amount of change in time is computed based on the amount of change and a frame rate.

2. A video evaluation device comprising:

an amount of change detection section configured to detect, based on a plurality of frame pictures included in an input video signal, an amount of change that indicates the degree of change between each of the frame pictures; and

an evaluation value calculation section configured to calculate an evaluation value relating to the smoothness in movement of the input video signal based on the amount of change and the time interval between each of the frame pictures and outputting the evaluation value to the outside wherein

the evaluation value calculation section computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the amount of change and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time,

the amount of change is a value based on a movement vector between the successive frame pictures, and

the amount of change in time is computed based on the amount of change and a frame rate.

3. The video evaluation device according toclaim 1, further comprising a feature value calculation section configured to calculate a movement feature value that indicates the feature of movement of the input video signal based on the amount of change, wherein the evaluation value calculation section calculates the evaluation value based on the movement feature value and the time interval between each of the frame pictures.

4. The video evaluation device according toclaim 3, wherein the evaluation value calculation section computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the movement feature value and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time.

5. The video evaluation device according toclaim 3, wherein:

the amount of change is a movement vector between each of the frame pictures; and

the movement feature value is a value calculated based on the magnitude of the movement vector.

6. A frame rate determination device comprising:

a frame rate generation section configured to generate a first frame rate;

an amount of change detection section configured to detect, based on a plurality of frame pictures included in an input video signal, an amount of change that indicates the degree of change between each of the frame pictures;

an evaluation value calculation section configured to calculate an evaluation value relating to the smoothness in movement of the input video signal based on the amount of change and the time interval between frame pictures in accordance with the first frame rate; and

a frame rate determination section configured to determine a second frame rate for carrying out the process of the input video signal using the evaluation value and outputting it to the outside, wherein

the evaluation value calculation section computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the amount of change and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time,

the amount of change is a value based on a difference in luminance value between pixels located at the same coordinates in the successive frame pictures, and

the amount of change in time is computed based on the amount of change and the first frame rate.

7. The frame rate determination device according toclaim 6, wherein the frame rate determination section compares the evaluation value with a predetermined set value and sets the second frame rate smaller than the first frame rate if the evaluation value is greater than the predetermined set value, and sets the second frame rate greater than the first frame rate when the evaluation value is smaller than the predetermined set value.

8. A video process device comprising:

a buffer section configured to store an input video signal;

a frame rate generation section configured to generate a first frame rate;

an amount of change detection section configured to detect, based on a plurality of frame pictures included in the input video signal, an amount of change that indicates the degree of change between each of the frame pictures;

an evaluation value calculation section configured to calculate an evaluation value relating to the smoothness in movement of the input video signal based on the amount of change and the time interval between frame pictures in accordance with the first frame rate;

a frame rate determination section configured to determine a second frame rate for carrying out the process of the input video signal using the evaluation value; and

a video process section configured to read the input video signal stored in the buffer section using the second frame rate and carrying out the video process, wherein

the evaluation value calculation section computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the amount of change and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time,

the amount of change is a value based on a difference in luminance value between pixels located at the same coordinates in the successive frame pictures, and

the amount of change in time is computed based on the amount of change and the first frame rate.

9. A video evaluation method comprising:

an amount of change detection step for detecting, based on a plurality of frame pictures included in an input video signal, an amount of change that indicates the degree of change between each of the frame pictures; and.

an evaluation value calculation step for calculating an evaluation value relating to the smoothness in movement of the input video signal based on the amount of change and the time interval between each of the frame pictures and outputting it to the outside, wherein

the evaluation value calculation step computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the amount of change and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time,

the amount of change is a value based on a difference in luminance value between pixels located at the same coordinates in the successive frame pictures, and

the amount of change in time is computed based on the amount of change and a frame rate.

10. A non-transitory computer-readable medium including computer program instructions, which when executed by a computer, cause the computer to perform a video evaluation method comprising:

detecting, based on a plurality of frame pictures included in an input video signal, an amount of change that indicates the degree of change between each of the frame pictures; and

calculating an evaluation value relating to the smoothness in movement of the input video signal based on the amount of change and the time interval between each of the frame pictures and outputting it to the outside, wherein

the step for calculating the evaluation value computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the amount of change and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time,

the amount of change is a value based on a movement vector between the successive frame pictures, and

the amount of change in time is computed based on the amount of change and a frame rate.

11. The video evaluation device according toclaim 2, further comprising a feature value calculation section configured to calculate a movement feature value that indicates the feature of movement of the input video signal based on the amount of change, wherein the evaluation value calculation section calculates the evaluation value based on the movement feature value and the time interval between each of the frame pictures.

12. The video evaluation device according toclaim 11, wherein the evaluation value calculation section computes an amount of change in time in accordance with the amount of change and the time interval between each of the frame pictures based on the movement feature value and the time interval between each of the frame pictures and calculates the evaluation value using the amount of change in time.

13. The video evaluation device according toclaim 4, wherein:

the amount of change is a movement vector between each of the frame pictures; and

the movement feature value is a value calculated based on the magnitude of the movement vector.

14. The video evaluation device according toclaim 11, wherein:

the amount of change is a movement vector between each of the frame pictures; and

the movement feature value is a value calculated based on the magnitude of the movement vector.

15. The video evaluation device according toclaim 12, wherein:

the amount of change is a movement vector between each of the frame pictures; and

the movement feature value is a value calculated based on the magnitude of the movement vector.

Images