Applying the conventional vertex transformations (seeSection “Vertex Transformations”) or any other transformations that are represented by matrices in shaders is usually accomplished by specifying the corresponding matrix in a uniform variable of the shader and then multiplying the matrix with a vector. There are, however, some differences in the details. Here, we discuss the transformation of points (i.e. 4D vectors with a 4th coordinates equal to 1), the transformation of directions (i.e. vectors in the strict sense: 3D vectors or 4D vectors with a 4th coordinate equal to 0), and the transformation of surface normal vectors (i.e. vectors that specify a direction that is orthogonal to a plane).

This section assumes some knowledge of the syntax of GLSL as described inSection “Vector and Matrix Operations”.

For points, transformations are usually represented by 4×4 matrices since they might include a translation by a 3D vectort in the 4th column:

    ${\displaystyle \text{ and }\mathbf{t}=\left[\begin{array}{c}{t}_1\\ t_2\\ t_3\end{array}\right]}$

(Projection matrices will also include additional values unequal to 0 in the last row.)

Three-dimensional points are represented by four-dimensional vectors with the 4th coordinate equal to 1:

${\displaystyle P=\left[\begin{array}{c}{p}_1\\ p_2\\ p_3\\ 1\end{array}\right]}$

In order to apply the transformation, the matrix${\displaystyle \mathrm{M}}$ is multiplied with the vector${\displaystyle P}$:

  ${\displaystyle =\left[\begin{array}{c}{a}_{1,1}{p}_1+a_{1,2}{p}_2+a_{1,3}{p}_3+t_1\\ a_{2,1}{p}_1+a_{2,2}{p}_2+a_{2,3}{p}_3+t_2\\ a_{3,1}{p}_1+a_{3,2}{p}_2+a_{3,3}{p}_3+t_3\\ 1\end{array}\right]}$

The GLSL code to apply a 4×4 matrix to a point represented by a 4D vector is straightforward:

mat4matrix;vec4point;vec4transformed_point=matrix*point;

Directions in three dimensions are represented either by a 3D vector or by a 4D vector with 0 as the fourth coordinate. (One can think of them as points at infinity; similar to a point at the horizon of which we cannot tell the position in space but only the direction in which to find it.)

In the case of a 3D vector, we can either transform it by multiplying it with a 3×3 matrix:

mat3matrix;vec3direction;vec3transformed_direction=matrix*direction;

or with a 4×4 matrix, if we convert the 3D vector to a 4D vector with a 4th coordinate equal to 0:

mat4matrix;vec3direction;vec3transformed_direction=vec3(matrix*vec4(direction,0.0));

Alternatively, the 4×4 matrix can also be converted to a 3×3 matrix.

On the other hand, a 4D vector can be multiplied directly with a 4×4 matrix. It can also be converted to a 3D vector in order to multiply it with a 3×3 matrix:

mat3matrix;vec4direction;// 4th component is 0vec4transformed_direction=vec4(matrix*vec3(direction),0.0);

Similarly to directions, surface normal vectors (or “normal vectors” for short) are represented by 3D vectors or 4D vectors with 0 as the 4th component. However, they transform differently. (The mathematical reason is that they represent something that is called a covector, covariant vector, one-form, or linear functional.)

To understand the transformation of normal vectors, consider the main feature of a surface normal vector: it is orthogonal to a surface. Of course, this feature should still be true under transformations, i.e. the transformed normal vector should be orthogonal to the transformed surface. If the surface is being represented locally by a tangent vector, this feature requires that a transformed normal vector is orthogonal to a transformed direction vector if the original normal vector is orthogonal to the original direction vector.

Mathematically spoken, a normal vectorn is orthogonal to a direction vectorv if their dot product is 0. It turns out that ifv is transformed by a 3×3 matrix${\displaystyle \mathrm{A}}$, the normal vector has to be transformed by thetransposed inverse of${\displaystyle \mathrm{A}}$:${\displaystyle ({\mathrm{A}}^{-1}{)}^T}$. We can easily test this by checking the dot product of the transformed normal vector${\displaystyle ({\mathrm{A}}^{-1}{)}^T}$n and the transformed direction vector${\displaystyle \mathrm{A}}$v:

${\displaystyle {\left({\mathrm{A}}^{-1}\right)}^T\mathbf{n}\cdot \mathrm{A}\mathbf{v}=}$  ${\displaystyle {\left({\left({\mathrm{A}}^{-1}\right)}^T\mathbf{n}\right)}^T\mathrm{A}\mathbf{v}=}$  ${\displaystyle \left({\mathbf{n}}^T{\left({\left({\mathrm{A}}^{-1}\right)}^T\right)}^T\right)\mathrm{A}\mathbf{v}=}$  ${\displaystyle \left({\mathbf{n}}^T{\mathrm{A}}^{-1}\right)\mathrm{A}\mathbf{v}=}$  ${\displaystyle {\mathbf{n}}^T{\mathrm{A}}^{-1}\mathrm{A}\mathbf{v}=}$  ${\displaystyle {\mathbf{n}}^T\mathbf{v}=}$  ${\displaystyle \mathbf{n}\cdot \mathbf{v}}$

In the first step we have used${\displaystyle \mathbf{a}\cdot \mathbf{b}}$ =${\displaystyle {\mathbf{a}}^T\mathbf{b}}$, then${\displaystyle (M\mathbf{a}{)}^T}$ =${\displaystyle {\mathbf{a}}^T{M}^T}$, then${\displaystyle ({\mathrm{M}}^T{)}^T=\mathrm{M}}$, then${\displaystyle {\mathrm{M}}^{-1}\mathrm{M}=\mathrm{I}\mathrm{d}}$ (i.e. the identity matrix).

The calculation shows that the dot product of the transformed vectors is in fact the same as the dot product of the original vectors; thus, the transformed vectors are orthogonal if and only if the original vectors are orthogonal. Just the way it should be.

Thus, in order to transform normal vectors in GLSL, thetransposed inverse matrix is often specified as a uniform variable (together with the original matrix for the transformation of directions and points) and applied as any other transformation:

mat3matrix_inverse_transpose;vec3normal;vec3transformed_normal=matrix_inverse_transpose*normal;

In the case of a 4×4 matrix, the normal vector can be cast to a 4D vector by appending 0:

mat4matrix_inverse_transpose;vec3normal;vec3transformed_normal=vec3(matrix_inverse_transpose*vec4(normal,0.0));

Alternatively, the matrix can be cast to a 3×3 matrix.

If the inverse matrix is known, the normal vector can be multiplied from the left to apply the transposed inverse matrix.In general, multiplying a transposed matrix with a vector can be easily expressed by putting the vector to the left of the matrix. The reason is that a vector-matrix product makes only sense for row vectors (i.e. transposed column vectors) and corresponds to a matrix-vector product of the transposed matrix with the corresponding column vector:

${\displaystyle {\mathbf{p}}^T\mathrm{A}={\left({\mathrm{A}}^T\mathbf{p}\right)}^T}$

Since GLSL makes no distinction between column and row vectors, the result is just a vector.

Thus, in order to multiply a normal vector with the transposed inverse matrix, we can multiply it from the left to the inverse matrix:

mat3matrix_inverse;vec3normal;vec3transformed_normal=normal*matrix_inverse;

In the case of multiplying a 4×4 matrix to a 4D normal vector (from the left or the right), it should be made sure that the 4th component of the resulting vector is 0. In fact, in several cases it is necessary to discard the computed 4th component (for example by casting the result to a 3D vector):

mat4matrix_inverse;vec4normal;vec4transformed_normal=vec4(vec3(normal*matrix_inverse),0.0);

Note that any normalization of the normal vector to unit length is not preserved by this transformation. Thus, normal vectors are often normalized to unit length after the transformation (e.g. with the built-in GLSL functionnormalize).

A special case arises when the transformation matrix${\displaystyle \mathrm{A}}$ is orthogonal. In this case, the inverse of${\displaystyle \mathrm{A}}$ is the transposed matrix; thus, the transposed of the inverse of${\displaystyle \mathrm{A}}$ is the twice transposed matrix, which is the original matrix, i.e. for a orthogonal matrix${\displaystyle \mathrm{A}}$:

${\displaystyle {\left({\mathrm{A}}^{-1}\right)}^T=}$  ${\displaystyle {\left({\mathrm{A}}^T\right)}^T=\mathrm{A}}$

Thus, in the case oforthogonal matrices, normal vectors are transformed with the same matrix as directions and points:

mat3matrix;// orthogonal matrixvec3normal;vec3transformed_normal=matrix*normal;

Sometimes it is necessary to apply the inverse transformation. In most cases, the best solution is to define another uniform variable for the inverse matrix and set the inverse matrix in the main application. The shader can then apply the inverse matrix like any other matrix. This is by far more efficient than computing the inverse in the shader.

There is, however, a special case: If the matrix${\displaystyle \mathrm{M}}$ is of the form presented above (i.e. the 4th row is (0,0,0,1)):

with an orthogonal 3×3 matrix${\displaystyle \mathrm{A}}$ (i.e. the row (or column) vectors of${\displaystyle \mathrm{A}}$ are normalized and orthogonal to each other; for example, this is usually the case for the view transformation, seeSection “Vertex Transformations”), then the inverse matrix is given by (because${\displaystyle {\mathrm{A}}^{-1}={\mathrm{A}}^T}$ for an orthogonal matrix${\displaystyle \mathrm{A}}$):

  

For the multiplication with a point${\displaystyle P}$ that is represented by the 4D vector${\displaystyle (p_x,p_y,p_z,1)}$ with the 3D vectorp${\displaystyle =(p_x,p_y,p_z)}$ we get:

  ${\displaystyle =\left[\begin{array}{c}{\mathrm{A}}^T\mathbf{p}-{\mathrm{A}}^T\mathbf{t}\\ 1\end{array}\right]}$  ${\displaystyle =\left[\begin{array}{c}{\mathrm{A}}^T(\mathbf{p}-\mathbf{t})\\ 1\end{array}\right]}$

Note that the vectort is just the 4th column of the matrix${\displaystyle \mathrm{M}}$, which can be conveniently accessed in GLSL:

mat4matrix;vec4last_column=matrix[3];// indices start with 0 in GLSL

As mentioned above, multiplying a transposed matrix with a vector can be easily expressed by putting the vector to the left of the matrix because a vector-matrix product makes only sense for row vectors (i.e. transposed column vectors) and corresponds to a matrix-vector product of the transposed matrix with the corresponding column vector:

${\displaystyle {\mathbf{p}}^T\mathrm{A}={\left({\mathrm{A}}^T\mathbf{p}\right)}^T}$

Using these features of GLSL, the term${\displaystyle A^T}$(p -t) is easily and efficiently implemented as follows (note that the 4th component of the result has to be set to 1 separately):

mat4matrix;// upper, left 3x3 matrix is orthogonal;// 4th row is (0,0,0,1)vec4point;// 4th component is 1vec4point_transformed_with_inverse=vec4(vec3((point-matrix[3])*matrix),1.0);

As in the case of points, the best way to transform a direction with the inverse matrix is usually to compute the inverse matrix in the main application and communicate it to a shader via another uniform variable.

The exception is an orthogonal 3×3 matrix${\displaystyle \mathrm{A}}$ (i.e. all rows (or columns) are normalized and orthogonal to each other) or a 4×4 matrix${\displaystyle \mathrm{M}}$ of the form

where${\displaystyle \mathrm{A}}$ is an orthogonal 3×3 matrix. In these cases, the inverse matrix${\displaystyle {\mathrm{A}}^{-1}}$ is equal to the transposed matrix${\displaystyle {\mathrm{A}}^T}$.

As discussed above, the best way in GLSL to multiply a vector with the transposed matrix is to multiply it from the left to the original matrix because this is interpreted as a product of a row vector with the original matrix, which corresponds to the product of the transposed matrix with the column vector:

${\displaystyle {\mathbf{p}}^T\mathrm{A}={\left({\mathrm{A}}^T\mathbf{p}\right)}^T}$

Thus, the transformation with the transposed matrix (i.e. the inverse in case of a orthogonal matrix) is written as:

mat4matrix;// upper, left 3x3 matrix is orthogonalvec4direction;// 4th component is 0vec4direction_transformed_with_inverse=vec4(vec3(direction*matrix),0.0);

Note that the 4th component of the result has to be set to 0 separately since the 4th component ofdirection * matrix is not meaningful for the transformation of directions. (It is, however, meaningful for the transformation of plane equations, which are not discussed here.)

The versions for 3x3 matrices and 3D vectors only require different cast operations between 3D and 4D vectors.

Suppose the inverse matrix${\displaystyle {\mathrm{M}}^{-1}}$ is available, but the transformation corresponding to${\displaystyle \mathrm{M}}$ is required. Moreover, we want to apply this transformation to a normal vector. In this case, we can just apply the transpose of the inverse by multiplying the normal vector from the left to the inverse matrix (as discussed above):

mat4matrix_inverse;vec3normal;vec3transformed_normal=vec3(vec4(normal,0.0)*matrix_inverse);

(or by casting the matrix).

Some frameworks (in particular the OpenGL compatibility profile but neither the OpenGL core profile nor OpenGL ES 2.x) provide several built-in uniforms to access certain vertex transformations in GLSL shaders. They should not be declared, but here are the declarations to specify their types:

uniformmat4gl_ModelViewMatrix;uniformmat4gl_ProjectionMatrix;uniformmat4gl_ModelViewProjectionMatrix;uniformmat4gl_TextureMatrix[gl_MaxTextureCoords];uniformmat3gl_NormalMatrix;// transpose of the inverse of the// upper left 3x3 matrix of gl_ModelViewMatrixuniformmat4gl_ModelViewMatrixInverse;uniformmat4gl_ProjectionMatrixInverse;uniformmat4gl_ModelViewProjectionMatrixInverse;uniformmat4gl_TextureMatrixInverse[gl_MaxTextureCoords];uniformmat4gl_ModelViewMatrixTranspose;uniformmat4gl_ProjectionMatrixTranspose;uniformmat4gl_ModelViewProjectionMatrixTranspose;uniformmat4gl_TextureMatrixTranspose[gl_MaxTextureCoords];uniformmat4gl_ModelViewMatrixInverseTranspose;uniformmat4gl_ProjectionMatrixInverseTranspose;uniformmat4gl_ModelViewProjectionMatrixInverseTranspose;uniformmat4gl_TextureMatrixInverseTranspose[gl_MaxTextureCoords];

The transformation of normal vectors is described in Section 2.12.2 of the “OpenGL 4.1 Compatibility Profile Specification” available at theKhronos OpenGL web site.

A more accessible description of the transformation of normal vectors is given in Appendix E of the free HTML version of the“OpenGL Programming Guide” availableonline.

The built-in uniforms of matrix transformations are described in Section 7.4.1 of the “OpenGL Shading Language 4.10.6 Specification” available at theKhronos OpenGL web site.

<GLSL Programming

• Book:GLSL Programming