This tutorial introducesimage-based lighting, in particulardiffuse (irradiance) environment mapping and its implementation with cube maps. (Unity'sLight Probes presumably work in a similar way but with dynamically rendered cube maps.)
This tutorial is based onSection “Reflecting Surfaces”. If you haven't read that tutorial, this would be a very good time to read it.
Consider the lighting of the sculpture in the image to the left. There is natural light coming through the windows. Some of this light bounces off the floor, walls and visitors before reaching the sculpture. Additionally, there are artificial light sources, and their light is also shining directly and indirectly onto the sculpture. How many directional lights and point lights would be needed to simulate this kind of complex lighting environment convincingly? At least more than a handful (probably more than a dozen) and therefore the performance of the lighting computations is challenging.
This problem is addressed by image-based lighting. For static lighting environments that are described by an environment map, e.g. a cube map, image-based lighting allows us to compute the lighting by an arbitrary number of light sources with a single texture lookup in a cube map (seeSection “Reflecting Surfaces” for a description of cube maps). How does it work?
In this section we focus on diffuse lighting. Assume that every texel (i.e. pixel) of a cube map acts as a directional light source. (Remember that cube maps are usually assumed to be infinitely large such that only directions matter, but positions don't.) The resulting lighting for a given surface normal direction can be computed as described inSection “Diffuse Reflection”. It's basically the cosine between the surface normal vectorN and the vector to the light sourceL:
Since the texels are the light sources,L is just the direction from the center of the cube to the center of the texel in the cube map. A small cube map with 32×32 texels per face has already 32×32×6 = 6144 texels. Adding the illumination by thousands of light sources is not going to work in real time. However, for a static cube map we can compute the diffuse illumination for all possible surface normal vectorsN in advance and store them in a lookup table. When lighting a point on a surface with a specific surface normal vector, we can then just look up the diffuse illumination for the specific surface normal vectorN in that precomputed lookup table.
Thus, for a specific surface normal vectorN we add (i.e. integrate) the diffuse illumination by all texels of the cube map. We store the resulting diffuse illumination for this surface normal vector in a second cube map (the “diffuse irradiance environment map” or “diffuse environment map” for short). This second cube map will act as a lookup table, where each direction (i.e. surface normal vector) is mapped to a color (i.e. diffuse illumination by potentially thousands of light sources). The fragment shader is therefore really simple (this one could use the vertex shader fromSection “Reflecting Surfaces”):
float4frag(vertexOutputinput):COLOR{returntexCUBE(_Cube,input.normalDir);}
It is just a lookup of the precomputed diffuse illumination using the surface normal vector of the rasterized surface point. However, the precomputation of the diffuse environment map is somewhat more complicated as described in the next section.
This section presents some C# code to illustrate the computation of cube maps for diffuse (irradiance) environment maps. In order to use it in Unity, chooseCreate > C# script in theProject Window and call is "ComputeDiffuseEnvironmentMap". Then open the script in Unity's text editor, copy the C# code into it, and attach the script to the game object that has a material with the shader presented below. When a new readable cube map of sufficiently small dimensions is specified for the shader property_OriginalCube (which is labeledEnvironment Map in the shader user interface), the script will update the shader property_Cube (i.e.Diffuse Environment Map in the user interface) with a corresponding diffuse environment map. Note that the cube map has to be "readable," i.e. you have to checkReadable in theInspector when creating the cube map. Also note that you should use small cube maps of face dimensions 32×32 or smaller because the computation time tends to be very long for larger cube maps. Thus, when creating a cube map in Unity, make sure to choose a sufficiently small size.
The script includes only a handful of functions:Awake() initializes the variables;Update() takes care of communicating with the user and the material (i.e. reading and writing shader properties);computeFilteredCubemap() does the actual work of computing the diffuse environment map; andgetDirection() is a small utility function forcomputeFilteredCubemap() to compute the direction associated with each texel of a cube map. Note thatcomputeFilteredCubemap() not only integrates the diffuse illumination but also avoids discontinuous seams between faces of the cube map by setting neighboring texels along the seams to the same averaged color.
Make sure to call the C# script file "ComputeDiffuseEnvironmentMap".
usingUnityEngine;usingUnityEditor;usingSystem.Collections;[ExecuteInEditMode]publicclassComputeDiffuseEnvironmentMap:MonoBehaviour{publicCubemaporiginalCubeMap;// environment map specified in the shader by the user//[System.Serializable]// avoid being deleted by the garbage collector,// and thus leakingprivateCubemapfilteredCubeMap;// the computed diffuse irradience environment mapprivatevoidUpdate(){CubemaporiginalTexture=null;try{originalTexture=GetComponent<Renderer>().sharedMaterial.GetTexture("_OriginalCube")asCubemap;}catch(System.Exception){Debug.LogError("'_OriginalCube' not found on shader. "+"Are you using the wrong shader?");return;}if(originalTexture==null)// did the user set "none" for the map?{if(originalCubeMap!=null){GetComponent<Renderer>().sharedMaterial.SetTexture("_Cube",null);originalCubeMap=null;filteredCubeMap=null;return;}}elseif(originalTexture==originalCubeMap&&filteredCubeMap!=null&&GetComponent<Renderer>().sharedMaterial.GetTexture("_Cube")==null){GetComponent<Renderer>().sharedMaterial.SetTexture("_Cube",filteredCubeMap);// set the computed// diffuse environment map in the shader}elseif(originalTexture!=originalCubeMap||filteredCubeMap!=GetComponent<Renderer>().sharedMaterial.GetTexture("_Cube")){if(EditorUtility.DisplayDialog("Processing of Environment Map","Do you want to process the cube map of face size "+originalTexture.width+"x"+originalTexture.width+"? (This will take some time.)","OK","Cancel")){if(filteredCubeMap!=GetComponent<Renderer>().sharedMaterial.GetTexture("_Cube")){if(GetComponent<Renderer>().sharedMaterial.GetTexture("_Cube")!=null){DestroyImmediate(GetComponent<Renderer>().sharedMaterial.GetTexture("_Cube"));// clean up}}if(filteredCubeMap!=null){DestroyImmediate(filteredCubeMap);// clean up}originalCubeMap=originalTexture;filteredCubeMap=computeFilteredCubeMap();//computes the diffuse environment mapGetComponent<Renderer>().sharedMaterial.SetTexture("_Cube",filteredCubeMap);// set the computed// diffuse environment map in the shaderreturn;}else{originalCubeMap=null;filteredCubeMap=null;GetComponent<Renderer>().sharedMaterial.SetTexture("_Cube",null);GetComponent<Renderer>().sharedMaterial.SetTexture("_OriginalCube",null);}}}// This function computes a diffuse environment map in// "filteredCubemap" of the same dimensions as "originalCubemap"// by integrating -- for each texel of "filteredCubemap" --// the diffuse illumination from all texels of "originalCubemap"// for the surface normal vector corresponding to the direction// of each texel of "filteredCubemap".privateCubemapcomputeFilteredCubeMap(){CubemapfilteredCubeMap=newCubemap(originalCubeMap.width,originalCubeMap.format,true);intfilteredSize=filteredCubeMap.width;intoriginalSize=originalCubeMap.width;// Compute all texels of the diffuse environment cube map// by itterating over all of themfor(intfilteredFace=0;filteredFace<6;filteredFace++)// the six sides of the cube{for(intfilteredI=0;filteredI<filteredSize;filteredI++){for(intfilteredJ=0;filteredJ<filteredSize;filteredJ++){Vector3filteredDirection=getDirection(filteredFace,filteredI,filteredJ,filteredSize).normalized;floattotalWeight=0.0f;Vector3originalDirection;Vector3originalFaceDirection;floatweight;ColorfilteredColor=newColor(0.0f,0.0f,0.0f);// sum (i.e. integrate) the diffuse illumination// by all texels in the original environment mapfor(intoriginalFace=0;originalFace<6;originalFace++){originalFaceDirection=getDirection(originalFace,1,1,3).normalized;//the normal vector of the facefor(intoriginalI=0;originalI<originalSize;originalI++){for(intoriginalJ=0;originalJ<originalSize;originalJ++){originalDirection=getDirection(originalFace,originalI,originalJ,originalSize);// direction to the texel// (i.e. light source)weight=1.0f/originalDirection.sqrMagnitude;// take smaller size of more// distant texels into accountoriginalDirection=originalDirection.normalized;weight=weight*Vector3.Dot(originalFaceDirection,originalDirection);// take tilt of texel compared// to face into accountweight=weight*Mathf.Max(0.0f,Vector3.Dot(filteredDirection,originalDirection));// directional filter// for diffuse illuminationtotalWeight=totalWeight+weight;// instead of analytically// normalization, we just normalize// to the potential max illuminationfilteredColor=filteredColor+weight*originalCubeMap.GetPixel((CubemapFace)originalFace,originalI,originalJ);// add the// illumination by this texel}}}filteredCubeMap.SetPixel((CubemapFace)filteredFace,filteredI,filteredJ,filteredColor/totalWeight);// store the diffuse illumination of this texel}}}// Avoid seams between cube faces: average edge texels// to the same color on each side of the seamintmaxI=filteredCubeMap.width-1;for(inti=0;i<maxI;i++){setFaceAverage(reffilteredCubeMap,0,i,0,2,maxI,maxI-i);setFaceAverage(reffilteredCubeMap,0,0,i,4,maxI,i);setFaceAverage(reffilteredCubeMap,0,i,maxI,3,maxI,i);setFaceAverage(reffilteredCubeMap,0,maxI,i,5,0,i);setFaceAverage(reffilteredCubeMap,1,i,0,2,0,i);setFaceAverage(reffilteredCubeMap,1,0,i,5,maxI,i);setFaceAverage(reffilteredCubeMap,1,i,maxI,3,0,maxI-i);setFaceAverage(reffilteredCubeMap,1,maxI,i,4,0,i);setFaceAverage(reffilteredCubeMap,2,i,0,5,maxI-i,0);setFaceAverage(reffilteredCubeMap,2,i,maxI,4,i,0);setFaceAverage(reffilteredCubeMap,3,i,0,4,i,maxI);setFaceAverage(reffilteredCubeMap,3,i,maxI,5,maxI-i,maxI);}// Avoid seams between cube faces:// average corner texels to the same color// on all three faces meeting in one cornersetCornerAverage(reffilteredCubeMap,0,0,0,2,maxI,maxI,4,maxI,0);setCornerAverage(reffilteredCubeMap,0,maxI,0,2,maxI,0,5,0,0);setCornerAverage(reffilteredCubeMap,0,0,maxI,3,maxI,0,4,maxI,maxI);setCornerAverage(reffilteredCubeMap,0,maxI,maxI,3,maxI,maxI,5,0,maxI);setCornerAverage(reffilteredCubeMap,1,0,0,2,0,0,5,maxI,0);setCornerAverage(reffilteredCubeMap,1,maxI,0,2,0,maxI,4,0,0);setCornerAverage(reffilteredCubeMap,1,0,maxI,3,0,maxI,5,maxI,maxI);setCornerAverage(reffilteredCubeMap,1,maxI,maxI,3,0,0,4,0,maxI);filteredCubeMap.Apply();//apply all SetPixel(..) commandsreturnfilteredCubeMap;}privatevoidsetFaceAverage(refCubemapfilteredCubeMap,inta,intb,intc,intd,inte,intf){Coloraverage=(filteredCubeMap.GetPixel((CubemapFace)a,b,c)+filteredCubeMap.GetPixel((CubemapFace)d,e,f))/2.0f;filteredCubeMap.SetPixel((CubemapFace)a,b,c,average);filteredCubeMap.SetPixel((CubemapFace)d,e,f,average);}privatevoidsetCornerAverage(refCubemapfilteredCubeMap,inta,intb,intc,intd,inte,intf,intg,inth,inti){Coloraverage=(filteredCubeMap.GetPixel((CubemapFace)a,b,c)+filteredCubeMap.GetPixel((CubemapFace)d,e,f)+filteredCubeMap.GetPixel((CubemapFace)g,h,i))/3.0f;filteredCubeMap.SetPixel((CubemapFace)a,b,c,average);filteredCubeMap.SetPixel((CubemapFace)d,e,f,average);filteredCubeMap.SetPixel((CubemapFace)g,h,i,average);}privateVector3getDirection(intface,inti,intj,intsize){switch(face){case0:returnnewVector3(0.5f,-((j+0.5f)/size-0.5f),-((i+0.5f)/size-0.5f));case1:returnnewVector3(-0.5f,-((j+0.5f)/size-0.5f),((i+0.5f)/size-0.5f));case2:returnnewVector3(((i+0.5f)/size-0.5f),0.5f,((j+0.5f)/size-0.5f));case3:returnnewVector3(((i+0.5f)/size-0.5f),-0.5f,-((j+0.5f)/size-0.5f));case4:returnnewVector3(((i+0.5f)/size-0.5f),-((j+0.5f)/size-0.5f),0.5f);case5:returnnewVector3(-((i+0.5f)/size-0.5f),-((j+0.5f)/size-0.5f),-0.5f);default:returnVector3.zero;}}}
As promised, the actual shader code is very short; the vertex shader is a reduced version of the vertex shader ofSection “Reflecting Surfaces”:
Shader"Cg shader with image-based diffuse lighting"{Properties{_OriginalCube("Environment Map",Cube)=""{}_Cube("Diffuse Environment Map",Cube)=""{}}SubShader{Pass{CGPROGRAM#pragma vertex vert#pragma fragment frag#include"UnityCG.cginc"// User-specified uniformsuniformsamplerCUBE_Cube;structvertexInput{float4vertex:POSITION;float3normal:NORMAL;};structvertexOutput{float4pos:SV_POSITION;float3normalDir:TEXCOORD0;};vertexOutputvert(vertexInputinput){vertexOutputoutput;float4x4modelMatrixInverse=unity_WorldToObject;// multiplication with unity_Scale.w is unnecessary// because we normalize transformed vectorsoutput.normalDir=normalize(mul(float4(input.normal,0.0),modelMatrixInverse).xyz);output.pos=UnityObjectToClipPos(input.vertex);returnoutput;}float4frag(vertexOutputinput):COLOR{returntexCUBE(_Cube,input.normalDir);}ENDCG}}}
The shader and script above are sufficient to compute diffuse illumination by a large number of static, directional light sources. But what about the specular illumination discussed inSection “Specular Highlights”, i.e.:
First, we have to rewrite this equation such that it depends only on the direction to the light sourceL and the reflected view vectorR:
With this equation, we can compute a lookup table (i.e. a cube map) that contains the specular illumination by many light sources for any reflected view vectorR. In order to look up the specular illumination with such a table, we just need to compute the reflected view vector and perform a texture lookup in a cube map. In fact, this is exactly what the shader code ofSection “Reflecting Surfaces” does. Thus, we actually only need to compute the lookup table.
It turns out that the JavaScript code presented above can be easily adapted to compute such a lookup table. All we have to do is to change the line
weight=weight*Mathf.Max(0.0,Vector3.Dot(filteredDirection,originalDirection));// directional filter for diffuse illumination
to
weight=weight*Mathf.Pow(Mathf.Max(0.0,Vector3.Dot(filteredDirection,originalDirection)),50.0);// directional filter for specular illumination
where50.0 should be replaced by a variable for. This allows us to compute lookup tables for any specific shininess. (The same cube map could be used for varying values of the shininess if the mipmap-level was specified explicitly using thetextureCubeLod instruction in the shader; however, this technique is beyond the scope of this tutorial.)
Congratulations, you have reached the end of a rather advanced tutorial! We have seen:
If you still want to know more