Theory of Deep Samples

This document derives the techniques for splitting and combining twonon-solid samples of equal depth and thickness. These should should beused by deep image “flattening” algorithms to compute the combinedcolour of two samples. The formulas are defined in the documentInterpreting OpenEXR Deep Pixels. This document derives those formulas,and is for information only.

Definitions

symbol

description

\(r\)

ratio of length of original sample to a subsample:\(0\le r \le 1\)

\(\mathbf{C_i}\)

Colour of sample\(i\)

\(\mathbf{C'_i}\)

Colour of a subsample of\(i\)

\(\alpha_i\)

alpha value of sample\(i\)

\(\alpha'_i\)

alpha value of a subsample of\(i\)

\(T_i\)

transparency/transmission of sample\(i\);\(T_a=1-\alpha_a\)

\(\mathbf{c_i}\)

colour per unit length (‘instantaneous colour’) of\(i\)

\(t_i\)

transparency per unit length (‘instantaneous transparency’) of\(i\)

Subscripts\(_a\) and\(_b\) refer to the input samples;subscript\(_c\) refers to the combined output sample

Sample Model

Many deep compositing operations require subdividing a sample into twosubsamples, or else clipping a sample at a given point. When thishappens, the alpha value of the subsample(s) must be recomputed, and thecolour will also change. To derive an equation for computing the newalpha, the amount of light attenuated by each subsample must beconsidered:

Light passing through multiple objects from left to right. Each object attenuates a fraction t of the light

Light passing through multiple objects from left to right. Each object attenuates a fraction t of the light

In this case, light passes through two objects, from left to right. Each objects attenuates the light. Light passing through any light absorbing object will by attenuated according to thetransparency\(t\) of that object. Thus, if\(i\) is the amount of light entering the object, the light\(o\) leaving the object is given by

(1)\[o=t\cdot i\]

Where light passes through two objects, the light is attenuated twice (the output from the first is fed into second)

(2)\[o=t_2\cdot(t_1\cdot i)\]

where\(t_2\) and\(t_1\) are the transmission of theindividual samples. In general, if the transmission\(t\) of allobjects are the same, the light emerging from\(n\) adjacent samplesis given by theBeer-Lambert law:

\[o=i\cdot t^n\]
(3)\[T=\frac{o}{i}=t^n\]

where\(T\) is the total transmission. If\(t\) is consideredthetransmission per unit length and\(n\) the length, theequation is continuous, applying also for non-integer values of\(n\)[1].

An OpenEXR deep sample can then be modelled as an object of a specifiedlength\(n\) which absorbs light according to equation Beer’s law(3)

Alpha vs Transparency

OpenEXR images use alpha\(\alpha\) instead of transparency.\(T=1-\alpha\): if\(\alpha=1\) (so\(T=0\)) then all lightis absorbed; if\(\alpha=0\) (so\(T=1\)), then all light istransmitted and the material is transparent. Substituting\(T\) into(1) and(2) combined alpha of two objects is given by thescreen compositing operation:

(4)\[\alpha_c = 1-(1-\alpha_1)(1-\alpha_2)\]

where\(\alpha_c\) is the combined alpha value, and\(\alpha_1\) and\(\alpha_2\) are the alpha values of the two samples.

Throughout this document,\(T\) is preferred to\(\alpha\) whereit gives simpler equations.

Computing\(t\)

OpenEXR images samples store the\(\alpha\) of the entire sample.This gives us the total transmission of the sample, not thetransmission per unit length\(t\). The total transmission\(T\)and the sample length\(n\) can be used to compute the transmission per unit length:

(5)\[t=T^\frac{1}{n}\]

Sample Properties

The following are assumptions made by the formulas derived here:

  • Samples haveconstant optical density and colour: if a sample haslength\(n\) and a subsample of length\(n'\) is extractedfrom it, the RGBA colour of the subsample will be the same regardlessfrom where in the sample it is extracted. In particular, if a sampleis split into\(k\) subsamples of equal length, each subsamplewill have the same RGBA colour.

  • Sample attenuation isnon-scattering andpixel independent:light travelling through the sample is either absorbed ortransmitted; it is not reflected back down the sample or scatteredinto neighbouring samples. Scattering causes point lights to appearblurred when passing through fog, and also tends to make fog lookmore optically dense than it really is, since detail is lost veryquickly, even though light is being transmitted. This effect is notmodelled with OpenEXR volumetric samples, and must be approximated byincluding the scattered light within the volume or applied as apost-process. Since the light attenuation profile throughout a pixelwill not follow the Beer-Lambert equation(3), extra samplesmust be used to model the pixel.

  • Sample behaviour isunit andscale independent: if a sampleis divided into\(k\) subsamples of equal length, then scalingthe depth channels\(z_\textsf{front}\) and\(z_\textsf{back}\) of the deep image will not change thecolour of the samples. When merging two deep images, prescaling thedepth of each image by the same amount then merging the images isidentical to merging the original images then scaling the depth ofthe result. This scaling property allows any unit to be used to storedepth, and the unit to be changed by scaling the depth channelswithout modification of the RGBA channels.

  • Sample behaviour isposition independent: moving a sample indepth will not change the RGBA values obtained by subdividing it. Ifa sample is\(10\) units long and a subsample of\(2\) unitsextracted from it, the subsample will have the same RGBA valuesregardless of the position in depth of the sample. Shifting twoimages by adding a constant\(c\) to the depth channels of eachimage then merging them is identical to merging the original imagesthen shifting the result.

  • Sample splitting isnon-destructive: using the equations derivedhere, the samples of a pixel can be arbitrarily subdivided.Recombining the subsamples with theover compositing operation willyield the RGBA colour of the original sample (assuming sufficientnumerical precision).

Sample Splitting

Alpha of a Subsample

When a subsample of length\(n'\) is extracted from an originalsample of length\(n\), the alpha value\(\alpha'\) forsubsample must be computed.

The total transmission\(T'=t^{n'}\). Substituting for\(t\)computed for the entire sample length using(5) gives:

\[\begin{split}\begin{aligned}T'&=\left(T^\frac{1}{n}\right)^{n'} \nonumber\\T'&=T^\frac{n'}{n}\end{aligned}\end{split}\]

Here,\(n\) refers to thelength of the original sample, butsamples are specified in OpenEXR with a front and back depth. Thus, if asample\(T\) with front and back depths\(z_\textsf{front}\)and\(z_\textsf{back}\) is split at point\(z\) in space,the transmission\(T'\) of the front subsample is given by

\[\begin{split}\begin{aligned}T'&=T^\frac{z-z_\textrm {front}}{z_\textrm {back} - z_\textrm {front}}\\\alpha'&=1-\left(1-\alpha\right)^\frac{z-z_\textrm{front}}{z_\textrm {back} -z_\textrm{front}}\end{aligned}\end{split}\]

Since\(({z-z_\textrm{front}})/({z_\textsf{back} - z_\textsf{front}})\)is theratio of the original sample length to the extracted samplelength, we can substitute this ratio for\(r\) for the remainder.This is equivalent to considering subdivision of a sample of unitlength.

Colour of a Subsample

A sample with RGB information is modelled as a cylinder. The alphachannel(s) of the sample are represented by some non-scattering,absorbent material within it, which attenuates the light passing throughthe sample. The colour channel(s) of the sample is represented byinfinitely many light sources, which emit coloured light (but don’tabsorb anything), with identical intensities\(\mathbf{c}\), evenlyspaced through the sample:

Model for a sample with colour and alpha.

Model for a sample with colour and alpha. The grey box is lightabsorbent material with transparency\(t\); the red dots arelight sources each of which has colour\(\mathbf{c}\).

A light source at distance\(x\) into the sample will be attenuatedby the amount of absorber between it and the front of the sample.According to Beer-Lambert, the total transmission of that part of thesample\(t\) is given by

\[\begin{split}\begin{aligned}t'&=T^x\\\alpha'&=1-(1-\alpha)^x\end{aligned}\end{split}\]

The light which reaches the front of the sample from the individuallight source at distance\(x\) is\(\mathbf{c} T^{x}\). If therewere a finite number\(N\) sources, the total light reaching thefront of the sample would be:

\[\mathbf{C} = \sum_{k=0}^{N}\mathbf{c}{T}^{k/N}\]

where\(k/N\) is the position of light\(k\), equal to\(x\). As\(N\) tends to infinity, this becomes

(6)\[\begin{split}\begin{aligned}\mathbf{C} &= \int_0^1\mathbf{c}{T}^x {\mathop{}\!\textrm{d}}x\\&=\left.\frac{\mathbf {c}{T}^x}{\log(T)}\right|_{x=0}^{x=1} \nonumber\\&=\frac{\mathbf{c}(T-1)}{\log(T)}\end{aligned}\end{split}\]

The value\(\mathbf{C}\) is the sample’s RGB value, as stored in theOpenEXR image. Rearranging gives us the colour of eachlight (or perhaps the ‘instantaneous colour’ of the sample)

(7)\[\mathbf{c} = \frac{\mathbf{C}\log(T)}{T-1}\]

If the entire sample is reduced to a subsample length\(r\), only asubsection of lights are included, and we must compute the new RGBcolour\(\mathbf{C'}\) for the subsample. Hence:

\[\begin{split}\begin{aligned}\mathbf{C'} &= \int_0^{r}\mathbf{c}{T}^x {\mathop{}\!\textrm{d}}x \\&=\frac{\mathbf{c}(T^{r}-1)}{\log(T)}\end{aligned}\end{split}\]

Substituting for\(\mathbf{c}\) from(7) gives

\[\begin{split}\begin{aligned}\mathbf{C'}&=\frac{\mathbf{C}\log(T)}{T-1}\frac{(T^{r}-1)}{\log(T)} \\&=\mathbf{C}\frac{(T^{r}-1)}{T-1}\end{aligned}\end{split}\]

Noting that\(T-1=-\alpha\) and\(T^{r}-1=T'-1=-\alpha'\) andmultiplying top and bottom by\(-1\) gives:

(8)\[\mathbf{C'}=\mathbf{C}\frac{\alpha'}{\alpha}\]

This is exactly equivalent tounpremultiplying the colour by theoriginal alpha value, computing the new alpha value, andpremultiplying by the new value.

Splitting Transparent Samples

When\(\alpha=0\) (or alternatively,\(T=1\)), unpremultiplying in(8) would require a division by zero.A special case is derived for\(\alpha=0\). The sample modelof the colour provided by many light sources still applies, but thereis now no attenuation. Therefore, each light contributes equally to theobserved colour\(\mathbf{C}\). The ‘instantaneous’ colour equalsthe final colour, (assuming sample length is normalised to 1)

(9)\[\begin{split}\begin{aligned}\mathbf{C}&=\int_0^1 \mathbf{c} {\mathop{}\!\textrm{d}}x\\\mathbf{C}&=\mathbf{c}\end{aligned}\end{split}\]

A subsection\(\mathbf{C'}\) of length\(r\) is given by

\[\begin{split}\begin{aligned}\mathbf{C'}&=\int_0^r \mathbf{c} {\mathop{}\!\textrm{d}}x \nonumber \\\mathbf{C'}&=r\mathbf{C}\end{aligned}\end{split}\]

That is, if\(\alpha=0\), then scaling a sample’s length by\(r\) scales the colour by the same amount.

Sample Merging

Now, consider combining two samples\(a\) and\(b\) together.This operation is required when “tidying” a deep sample list, which isessential before flattening a deep image into a regular one. We assumethat\(z_{\textsf{front}_a}=z_{\textsf{front}_b}\) and\(z_{\textsf{back}_a}=z_{\textsf{back}_b}\) so the lengthsof each sample are the same, and they fully overlap in space. If this isnot the case, the samples should be subdivided and those subsamplesmerged separately as described in the document “Interpreting OpenEXRDeep Samples”

The same sample model using discrete light sources is used. However, ateach location, there are now two light sources,\(\mathbf{c_a}\) and\(\mathbf{c_b}\), and the light is attenuated by both\(t_a\)and\(t_b\). We can treat this as being attenuated first by\(t_b\), then by\(t_a\), so the total light reaching the end ofthe sample is:

\[\mathbf{c_c}=(\mathbf{c_a}+\mathbf{c_b})(T_{a}T_b)^{x}\]

By substitution into(6) and then substituting for\(\mathbf{c_a}\) and\(\mathbf{c_b}\) from(7) gives

\[\begin{split}\begin{aligned}\mathbf{C_c}&=\frac{(\mathbf{c_a}+\mathbf{c_b})(T_aT_b-1)}{\log(T_aT_b)}\\\mathbf{C_c}&=\frac{\left(\frac{\mathbf{C_a}\log(T_a)}{T_a-1} +\frac{\mathbf{C_b}\log(T_b)}{T_b-1}\right)\left(T_aT_b-1\right)}{\log(T_aT_b)}\\\mathbf{C_c}&=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{-\alpha_a} +\frac{\mathbf{C_b}\log(1-\alpha_b)}{-\alpha_b}\right)\left((1-\alpha_a)(1-\alpha_b)-1\right)}{\log\left((1-\alpha_a)(1-\alpha_b)\right)}\\\end{aligned}\end{split}\]

Multiplying both parts of the top line by -1 gives

(10)\[\mathbf{C_c}=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} +\frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b}\right)\left(1-(1-\alpha_a)(1-\alpha_b)\right)}{\log\left((1-\alpha_a)(1-\alpha_b)\right)}\]

Note that the combined transmission is given by\(T_c=T_{a}T_{b}\),implying that the combined alpha follows the screen equation(6). Substituting into both top and bottom of(10)gives the formula for final colour:

(11)\[\mathbf{C_c}=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} +\frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b}\right)\alpha_c}{\log(1-\alpha_c)}\]

This is thepremultiplied combined colour. For the unpremultipliedcolour, the\(\alpha_c\) term can be omitted. This gives thetransmission weighted average of the unpremultiplied input colours.

Merging when One Sample is Transparent

Now suppose one of the samples (say,\(b\)) has no alpha, so\(\alpha_b=0\) and\(T_b=1\). From(9)\(\mathbf{c_b}=\mathbf{C_b}\). The combined colour\(\mathbf{C_c}\) is attenuated according to\(T_a\), thetransmission of\(a\) alone, and the combined colour is given by:

\[\mathbf{c_c} = (\mathbf{c_a}+\mathbf{C_b})(T_a)^{x}\]

and substituting into(6) gives:

\[\begin{split}\begin{aligned}\mathbf{C_c}&= \frac{(\mathbf{c_a}+\mathbf{C_b})(T_a-1)}{\log(T_a)}\nonumber\\&=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{-\alpha_a} +\mathbf{C_b}\right)\left((1-\alpha_a)-1\right)}{\log(1-\alpha_a)} \nonumber\\\mathbf{C_c}&=\frac{\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} -\mathbf{C_b}\right)\alpha_a}{\log(1-\alpha_a)}\end{aligned}\end{split}\]

Merging Two Transparent Samples

Where both samples are transparent, the colours simply add together:

\[\mathbf{C_c}=\mathbf{C_a}+\mathbf{C_b}\]

Merging Solid Samples

For numerical stability, we must give a sensible value for\(\mathbf{C_c}\) when\(\alpha_b=1\). Equation(11)is undefined, since\(\log(1-\alpha_b)=\log(0)=-\infty\). We followthe initial model of light sources in an absorbing material, but now theabsorbing material absorbs all light. Thus, we must be observing onlyits closest light source (that at\(x=0\)) in the solid sample: thesample will absorb light from its other sources. Thus, we can treat\(b\) as an infinitely thin ‘discrete’ sample at the front of thesample, which will block all light behind it; we can simply composite\(b\) over\(a\). Therefore, if\(\alpha_b=1\) and\(\alpha_a<1\), the total observed colour will be\(\mathbf{C_b}\). Transposing\(a\) and\(b\) gives\(\mathbf{C_c}=\mathbf{C_a}\) if\(\alpha_a=1\)

Following the definitions above, when\(\alpha_a=1\) and\(\alpha_b=1\), then\(\mathbf{C_c}=\mathbf{C_a}+\mathbf{C_b}\).However, for stability, it makes sense to define

\[\mathbf{C_c}=\frac{\mathbf{C_a}+\mathbf{C_b}}{2}\]

This approach gives more stable results when there is random samplingerror in the depth channel. Assume that the front depth of sample\(A\) differs by some small amount from that of\(B\), so that\(z_a=\delta+z_b\). Only the front sample will be visible: if\(\delta<0\), then\(\mathbf{C_c}=\mathbf{C_a}\); if\(\delta>0\), then\(\mathbf{C_c}=\mathbf{C_b}\). Setting\(\mathbf{C_c}=\mathbf{C_a}+\mathbf{C_b}\) when\(\delta=0\)gives an image which is (potentially) twice as bright. If\(\delta\)is random noise due to sampling, we will get a random pattern of brightpixels whenever\(\delta=0\). Using the mean colour seems a saferoption. However, this isnot associative: when combining three solidsamples\(d\),\(e\) and\(f\) does not give the sameresult as combining first\(d\) and\(e\), then combining theresult with\(f\).

Summary

Toclip a sample covering range\(z_\textsf{front}\)\(z_\textsf{back}\) into a subsample\(z_\textsf{front}\)\(z\), orextract a subsample oflength\(l'\) from a sample of length\(l\):

\[\begin{split}\begin{aligned}r&=\frac{z-z_\textsf{front}}{z_\textsf{back}-z_\textsf{front}} = \frac{l'}{l}\\\alpha' &= 1-(1-\alpha)^r\\\mathbf{C'} &=\begin{cases}r\mathbf{C},& \text{ if } \alpha=0\\\mathbf{C}\frac{\alpha'}{\alpha},& \text{ otherwise}\end{cases}\end{aligned}\end{split}\]

where\(\alpha\) and\(\alpha'\) are the alpha values of theoriginal sample and the subsample respectively, and\(\mathbf{C}\)and\(\mathbf{C'}\) are the colours of the original sample and thesubsample respectively

Tomerge two samples\(a\) and\(b\) into a combined sample\(c\):

\[\begin{split}\begin{aligned}\alpha_c &= 1-(1-\alpha_a)(1-\alpha_b)\\\mathbf{C_c} &=\begin{cases}\mathbf{C_a} + \mathbf{C_b}, & \text{if } \alpha_a=0 \text { and } \alpha_b=0\\\mathbf{C_a},& \text{if } \alpha_a=1 \text{ and } \alpha_b <1 \\\mathbf{C_b},& \text{if } \alpha_b=1 \text{ and } \alpha_a <1 \\(\mathbf{C_a} + \mathbf{C_b})/2, & \text{if } \alpha_a=1 \text { and } \alpha=1\\\alpha_a\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} - \mathbf{C_b}\right)/\log(1-\alpha_a),&\text {if } 0<\alpha_a<1 \text { and } \alpha_b=0 \\\alpha_b\left(\frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b} - \mathbf{C_a}\right)/\log(1-\alpha_b),&\text {if } 0<\alpha_b<1 \text { and } \alpha_a=0 \\\alpha_c\left(\frac{\mathbf{C_a}\log(1-\alpha_a)}{\alpha_a} +\frac{\mathbf{C_b}\log(1-\alpha_b)}{\alpha_b}\right)/\log(1-\alpha_c),& \text{otherwise}\end{cases}\end{aligned}\end{split}\]
[1]

Various forms of the Beer-Lambert law exist, using differentconstants. For example,\(o=i\cdot10^{-\epsilon l c}\), where\(\epsilon\) and\(c\) are the molar absorptivity andconcentration respectively. This is equivalent to(3) if\(t=10^{-\epsilon c}\) and\(l=n\)