This article has multiple issues. Please helpimprove it or discuss these issues on thetalk page.(Learn how and when to remove these messages) This articleneeds additional citations forverification. Please helpimprove this article byadding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Visual information fidelity" – news ·newspapers ·books ·scholar ·JSTOR(May 2017) (Learn how and when to remove this message) This article or sectionmay have beencopied and pasted from another location,possibly in violation ofWikipedia's copyright policy. Please review the source andremedy this by editing this article to remove any non-free copyrighted content and attributing free content correctly, or flagging the content for deletion. Please be sure that the supposed source of the copyright violation is not itself aWikipedia mirror.(January 2024) (Learn how and when to remove this message)

Visual information fidelity (VIF) is a full referenceimage quality assessment index based onnatural scene statistics and the notion of image information extracted by thehuman visual system.^[1] It was developed by Hamid R Sheikh andAlan Bovik at the Laboratory for Image and Video Engineering (LIVE) at theUniversity of Texas at Austin in 2006. It is deployed in the core of theNetflixVMAF video quality monitoring system, which controls the picture quality of all encoded videos streamed by Netflix.

Images and videos of thethree-dimensional visual environments come from a common class: the class of natural scenes. Natural scenes from a tiny subspace in the space of all possible signals, and researchers have developed sophisticated models to characterize these statistics. Most real-worlddistortion processes disturb these statistics and make the image or video signals unnatural. The VIF index employsnatural scene statistical (NSS) models in conjunction with adistortion (channel) model to quantify the information shared between the test and the reference images. Further, the VIF index is based on thehypothesis that this shared information is an aspect of fidelity that relates well with visual quality. In contrast to prior approaches based onhuman visual system (HVS) error-sensitivity and measurement of structure,^[2] this statistical approach used in aninformation-theoretic setting, yields a full reference (FR)quality assessment (QA) method that does not rely on any HVS or viewing geometry parameter, nor any constants requiring optimization, and yet is competitive with state of the art QA methods.^[3]

Specifically, the reference image is modeled as being the output of astochastic 'natural' source that passes through the HVS channel and is processed later by the brain. The information content of the reference image is quantified as being the mutual information between the input and output of the HVS channel. This is the information that the brain could ideally extract from the output of the HVS. The same measure is then quantified in the presence of an image distortion channel that distorts the output of the natural source before it passes through the HVS channel, thereby measuring the information that the brain could ideally extract from the test image. This is shown pictorially in Figure 1. The two information measures are then combined to form a visual information fidelity measure that relates visual quality to relative image information.

This articlemay be too technical for most readers to understand. Pleasehelp improve it tomake it understandable to non-experts, without removing the technical details.(January 2018) (Learn how and when to remove this message)

A Gaussian scale mixture (GSM) is used to statistically model thewavelet coefficients of a steerable pyramid decomposition of an image.^[4] The model is described below for a given subband of the multi-scale multi-orientation decomposition and can be extended to other subbands similarly. Let the wavelet coefficients in a given subband be${\displaystyle \mathcal{C}=\{{\overline{C}}_i:i\in \mathcal{I}\}}$ where${\displaystyle \mathcal{I}}$ denotes the set of spatial indices across the subband and each${\displaystyle {\overline{C}}_i}$ is an${\displaystyle M}$ dimensionalvector. The subband is partitioned into non-overlapping blocks of${\displaystyle M}$coefficients each, where each block corresponds to${\displaystyle {\overline{C}}_i}$. According to the GSM model,$${\displaystyle \mathcal{C}=\mathcal{S}\cdot \mathcal{U}=\{S_i{\overline{U}}_i:i\in \mathcal{I}\},}$$where${\displaystyle S_i}$ is a positivescalar and${\displaystyle {\overline{U}}_i}$ is a Gaussian vector with mean zero andco-variance${\displaystyle {\mathbf{C}}_U}$. Further the non-overlapping blocks are assumed to be independent of each other and that the random field${\displaystyle \mathcal{S}}$ is independent of${\displaystyle \mathcal{U}}$.

The distortion process is modeled using a combination of signalattenuation and additive noise in thewavelet domain. Mathematically, if${\displaystyle \mathcal{D}=\{{\overline{D}}_i:i\in \mathcal{I}\}}$ denotes the random field from a given subband of the distorted image,${\displaystyle \mathcal{G}=\{g_i:i\in \mathcal{I}\}}$ is adeterministic scalar field and${\displaystyle \mathcal{V}=\{{\overline{V}}_i:i\in \mathcal{I}\}}$, where${\displaystyle {\overline{V}}_i}$ is a zero mean Gaussian vector with co-variance${\displaystyle {\mathbf{C}}_V={\sigma }_v^2\mathbf{I}}$, then

${\displaystyle \mathcal{D}=\mathcal{G}\mathcal{C}+\mathcal{V}.}$

Further,${\displaystyle \mathcal{V}}$ is modeled to be independent of${\displaystyle \mathcal{S}}$ and${\displaystyle \mathcal{U}}$.

The duality of HVS models and NSS implies that several aspects of the HVS have already been accounted for in the source model. Here, the HVS is additionally modeled based on the hypothesis that the uncertainty in theperception of visual signals limits the amount of information that can be extracted from the source and distorted image. This source of uncertainty can be modeled asvisual noise in the HVS model. In particular, the HVS noise in a given subband of the wavelet decomposition is modeled as additive white Gaussian noise. Let${\displaystyle \mathcal{N}=\{{\overline{N}}_i:i\in \mathcal{I}\}}$ and${\displaystyle {\mathcal{N}}^{\prime }=\{{\overline{N}}_i^{\prime }:i\in \mathcal{I}\}}$ be random fields, where${\displaystyle {\overline{N}}_i}$ and${\displaystyle {\overline{N}}_i^{\prime }}$ are zero mean Gaussian vectors with co-variance${\displaystyle {\mathbf{C}}_N}$ and${\displaystyle {\mathbf{C}}_N^{\prime }}$. Further, let${\displaystyle \mathcal{E}}$ and${\displaystyle \mathcal{F}}$ denote the visual signal at the output of the HVS. Mathematically, we have${\displaystyle \mathcal{E}=\mathcal{C}+\mathcal{N}}$ and${\displaystyle \mathcal{F}=\mathcal{D}+{\mathcal{N}}^{\prime }}$. Note that${\displaystyle \mathcal{N}}$ and${\displaystyle {\mathcal{N}}^{\prime }}$ arerandom fields that are independent of${\displaystyle \mathcal{S}}$,${\displaystyle \mathcal{U}}$ and${\displaystyle \mathcal{V}}$.

This sectiondoes notcite anysources. Please helpimprove this section byadding citations to reliable sources. Unsourced material may be challenged andremoved.(January 2018) (Learn how and when to remove this message)

This articlemay be too technical for most readers to understand. Pleasehelp improve it tomake it understandable to non-experts, without removing the technical details.(January 2018) (Learn how and when to remove this message)

Let${\displaystyle {\overline{C}}^N=({\overline{C}}_1,{\overline{C}}_2,\dots ,{\overline{C}}^N)}$ denote the vector of all blocks from a given subband. Let${\displaystyle S^N,{\overline{D}}^N,{\overline{E}}^N}$ and${\displaystyle {\overline{F}}^N}$ be similarly defined. Let${\displaystyle s^N}$ denote themaximum likelihood estimate of${\displaystyle S^N}$ given${\displaystyle C^N}$ and${\displaystyle {\mathbf{C}}_U}$. The amount of information extracted from the reference is obtained as

${\displaystyle I({\overline{C}}^N;{\overline{E}}^N|{\overline{S}}^N=s^N)=\frac{1}{2}\sum _{i=1}^N{\log }_2\left(\frac{|s_i^2{\mathbf{C}}_U+{\sigma }_n^2\mathbf{I}|}{|{\sigma }_n^2\mathbf{I}|}\right),}$

while the amount of information extracted from the test image is given as

${\displaystyle I({\overline{C}}^N;{\overline{F}}^N|{\overline{S}}^N=s^N)=\frac{1}{2}\sum _{i=1}^N{\log }_2\left(\frac{|g_i^2{s}_i^2{\mathbf{C}}_U+({\sigma }_v^2+{\sigma }_n^2)\mathbf{I}|}{|({\sigma }_v^2+{\sigma }_n^2)\mathbf{I}|}\right).}$

Denoting the${\displaystyle N}$ blocks in subband${\displaystyle j}$ of the wavelet decomposition by${\displaystyle {\overline{C}}^{N,j}}$, and similarly for the other variables, the VIF index is defined as

${\displaystyle \text{VIF}=\frac{\sum _{j\in \text{subbands}}I({\overline{C}}^{N,j};{\overline{F}}^{N,j}\mid S^{N,j}=s^{N,j})}{\sum _{j\in \text{subbands}}I({\overline{C}}^{N,j};{\overline{E}}^{N,j}\mid S^{N,j}=s^{N,j})}.}$

The Spearman's rank-order correlation coefficient (SROCC) between the VIF index scores of distorted images on the LIVE Image Quality Assessment Database and the corresponding human opinion scores is evaluated to be 0.96.^{[citation needed]}

• Laboratory for Image and Video Engineering at the University of Texas
• An implementation of the VIF index
• LIVE Image Quality Assessment Database

• University of Texas System
• Film and video technology
• Image processing

• Articles with short description
• Short description is different from Wikidata
• Articles needing additional references from May 2017
• All articles needing additional references
• Copied and pasted articles and sections from January 2024
• All copied and pasted articles and sections
• Articles with multiple maintenance issues
• Wikipedia articles that are too technical from January 2018
• All articles that are too technical
• Articles needing additional references from January 2018
• All articles with unsourced statements
• Articles with unsourced statements from March 2021