| Part of a series of articles about | ||||||
| Calculus | ||||||
|---|---|---|---|---|---|---|
| ||||||
| ||||||
Specialized | ||||||
Incalculus, thechain rule is aformula that expresses thederivative of thecomposition of twodifferentiable functionsf andg in terms of the derivatives off andg. More precisely, if is the function such that for everyx, then the chain rule is, inLagrange's notation,or, equivalently,
The chain rule may also be expressed inLeibniz's notation. If a variablez depends on the variabley, which itself depends on the variablex (that is,y andz aredependent variables), thenz depends onx as well, via the intermediate variabley. In this case, the chain rule is expressed as andfor indicating at which points the derivatives have to be evaluated.
Inintegration, the counterpart to the chain rule is thesubstitution rule.
Intuitively, the chain rule states that knowing the instantaneous rate of change ofz relative toy and that ofy relative tox allows one to calculate the instantaneous rate of change ofz relative tox as the product of the two rates of change.
As put byGeorge F. Simmons: "If a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels 2 × 4 = 8 times as fast as the man."[1][2]
The relationship between this example and the chain rule is as follows. Letz,y andx be the (variable) positions of the car, the bicycle, and the walking man, respectively. The rate of change of relative positions of the car and the bicycle is Similarly, So, the rate of change of the relative positions of the car and the walking man is
The rate of change of positions is the ratio of the speeds, and the speed is the derivative of the position with respect to the time; that is,or, equivalently,which is also an application of the chain rule.
The chain rule seems to have first been used byGottfried Wilhelm Leibniz. He used it to calculate the derivative of as the composite of the square root function and the function. He first mentioned it in a 1676 memoir (with a sign error in the calculation).[3] The common notation of the chain rule is due to Leibniz.[4]Guillaume de l'Hôpital used the chain rule implicitly in hisAnalyse des infiniment petits. The chain rule does not appear in any ofLeonhard Euler's analysis books, even though they were written over a hundred years after Leibniz's discovery.[citation needed]. It is believed that the first "modern" version of the chain rule appears in Lagrange's 1797Théorie des fonctions analytiques; it also appears in Cauchy's 1823Résumé des Leçons données a L’École Royale Polytechnique sur Le Calcul Infinitesimal.[4]
The simplest form of the chain rule is for real-valued functions of onereal variable. It states that ifg is a function that is differentiable at a pointc (i.e. the derivativeg′(c) exists) andf is a function that is differentiable atg(c), then the composite function is differentiable atc, and the derivative is[5]The rule is sometimes abbreviated as
Ify =f(u) andu =g(x), then this abbreviated form is written inLeibniz notation as:
The points where the derivatives are evaluated may also be stated explicitly:
Carrying the same reasoning further, givenn functions with the composite function, if each function is differentiable at its immediate input, then the composite function is also differentiable by the repeated application of Chain Rule, where the derivative is (in Leibniz's notation):
The chain rule can be applied to composites of more than two functions. To take the derivative of a composite of more than two functions, notice that the composite off,g, andh (in that order) is the composite off withg ∘h. The chain rule states that to compute the derivative off ∘g ∘h, it is sufficient to compute the derivative off and the derivative ofg ∘h. The derivative off can be calculated directly, and the derivative ofg ∘h can be calculated by applying the chain rule again.[citation needed]
For concreteness, consider the functionThis can be decomposed as the composite of three functions:So that.
Their derivatives are:
The chain rule states that the derivative of their composite at the pointx =a is:
InLeibniz's notation, this is:or for short,The derivative function is therefore:
Another way of computing this derivative is to view the composite functionf ∘g ∘h as the composite off ∘g andh. Applying the chain rule in this manner would yield:
This is the same as what was computed above. This should be expected because(f ∘g) ∘h =f ∘ (g ∘h).
Sometimes, it is necessary to differentiate an arbitrarily long composition of the form. In this case, definewhere and when. Then the chain rule takes the formor, in the Lagrange notation,
The chain rule can be used to derive some well-known differentiation rules. For example, the quotient rule is a consequence of the chain rule and theproduct rule. To see this, write the functionf(x)/g(x) as the productf(x) · 1/g(x). First apply the product rule:
To compute the derivative of1/g(x), notice that it is the composite ofg with the reciprocal function, that is, the function that sendsx to1/x. The derivative of the reciprocal function is. By applying the chain rule, the last expression becomes:which is the usual formula for the quotient rule.
Suppose thaty =g(x) has aninverse function. Call its inverse functionf so that we havex =f(y). There is a formula for the derivative off in terms of the derivative ofg. To see this, note thatf andg satisfy the formula
And because the functions andx are equal, their derivatives must be equal. The derivative ofx is the constant function with value 1, and the derivative of is determined by the chain rule. Therefore, we have that:
To expressf' as a function of an independent variabley, we substitute forx wherever it appears. Then we can solve forf'.
For example, consider the functiong(x) =ex. It has an inversef(y) = lny. Becauseg′(x) =ex, the above formula says that
This formula is true wheneverg is differentiable and its inversef is also differentiable. This formula can fail when one of these conditions is not true. For example, considerg(x) =x3. Its inverse isf(y) =y1/3, which is not differentiable at zero. If we attempt to use the above formula to compute the derivative off at zero, then we must evaluate1/g′(f(0)). Sincef(0) = 0 andg′(0) = 0, we must evaluate 1/0, which is undefined. Therefore, the formula fails in this case. This is not surprising becausef is not differentiable at zero.
The chain rule forms the basis of theback propagation algorithm, which is used ingradient descent ofneural networks indeep learning (artificial intelligence).[6]
Faà di Bruno's formula generalizes the chain rule to higher derivatives. Assuming thaty =f(u) andu =g(x), then the first few derivatives are:
One proof of the chain rule begins by defining the derivative of the composite functionf ∘g, where we take thelimit of thedifference quotient forf ∘g asx approachesa:
Assume for the moment that does not equal for any near. Then the previous expression is equal to the product of two factors:
If oscillates neara, then it might happen that no matter how close one gets toa, there is always an even closerx such thatg(x) =g(a). For example, this happens neara = 0 for thecontinuous functiong defined byg(x) = 0 forx = 0 andg(x) =x2 sin(1/x) otherwise. Whenever this happens, the above expression is undefined because it involvesdivision by zero. To work around this, introduce a function as follows:We will show that the difference quotient forf ∘g is always equal to:
Wheneverg(x) is not equal tog(a), this is clear because the factors ofg(x) −g(a) cancel. Wheng(x) equalsg(a), then the difference quotient forf ∘g is zero becausef(g(x)) equalsf(g(a)), and the above product is zero because it equalsf′(g(a)) times zero. So the above product is always equal to the difference quotient, and to show that the derivative off ∘g ata exists and to determine its value, we need only show that the limit asx goes toa of the above product exists and determine its value.
To do this, recall that the limit of a product exists if the limits of its factors exist. When this happens, the limit of the product of these two factors will equal the product of the limits of the factors. The two factors areQ(g(x)) and(g(x) −g(a)) / (x −a). The latter is the difference quotient forg ata, and becauseg is differentiable ata by assumption, its limit asx tends toa exists and equalsg′(a).
As forQ(g(x)), notice thatQ is defined whereverf is. Furthermore,f is differentiable atg(a) by assumption, soQ is continuous atg(a), by definition of the derivative. The functiong is continuous ata because it is differentiable ata, and thereforeQ ∘g is continuous ata. So its limit asx goes toa exists and equalsQ(g(a)), which isf′(g(a)).
This shows that the limits of both factors exist and that they equalf′(g(a)) andg′(a), respectively. Therefore, the derivative off ∘g ata exists and equalsf′(g(a))g′(a).
Another way of proving the chain rule is to measure the error in the linear approximation determined by the derivative. This proof has the advantage that it generalizes to several variables. It relies on the following equivalent definition of differentiability at a point: A functiong is differentiable ata if there exists a real numberg′(a) and a functionε(h) that tends to zero ash tends to zero, and furthermoreHere the left-hand side represents the true difference between the value ofg ata and ata +h, whereas the right-hand side represents the approximation determined by the derivative plus an error term.
In the situation of the chain rule, such a functionε exists becauseg is assumed to be differentiable ata. Again by assumption, a similar function also exists forf atg(a). Calling this functionη, we haveThe above definition imposes no constraints onη(0), even though it is assumed thatη(k) tends to zero ask tends to zero. If we setη(0) = 0, thenη is continuous at 0.
Proving the theorem requires studying the differencef(g(a +h)) −f(g(a)) ash tends to zero. The first step is to substitute forg(a +h) using the definition of differentiability ofg ata:The next step is to use the definition of differentiability off atg(a). This requires a term of the formf(g(a) +k) for somek. In the above equation, the correctk varies withh. Setkh =g′(a)h +ε(h)h and the right hand side becomesf(g(a) +kh) −f(g(a)). Applying the definition of the derivative gives:To study the behavior of this expression ash tends to zero, expandkh. After regrouping the terms, the right-hand side becomes:Becauseε(h) andη(kh) tend to zero ash tends to zero, the first two bracketed terms tend to zero ash tends to zero. Applying the same theorem on products of limits as in the first proof, the third bracketed term also tends zero. Because the above expression is equal to the differencef(g(a +h)) −f(g(a)), by the definition of the derivativef ∘g is differentiable ata and its derivative isf′(g(a))g′(a).
The role ofQ in the first proof is played byη in this proof. They are related by the equation:The need to defineQ atg(a) is analogous to the need to defineη at zero.
Constantin Carathéodory's alternative definition of the differentiability of a function can be used to give an elegant proof of the chain rule.[7]
Under this definition, a functionf is differentiable at a pointa if and only if there is a functionq, continuous ata and such thatf(x) −f(a) =q(x)(x −a). There is at most one such function, and iff is differentiable ata thenf ′(a) =q(a).
Given the assumptions of the chain rule and the fact that differentiable functions and compositions of continuous functions are continuous, we have that there exist functionsq, continuous atg(a), andr, continuous ata, and such that,andTherefore,but the function given byh(x) =q(g(x))r(x) is continuous ata, and we get, for thisaA similar approach works for continuously differentiable (vector-)functions of many variables. This method of factoring also allows a unified approach to stronger forms of differentiability, when the derivative is required to beLipschitz continuous,Hölder continuous, etc. Differentiation itself can be viewed as thepolynomial remainder theorem (the littleBézout theorem, or factor theorem), generalized to an appropriate class of functions.[citation needed]
The full generalization of the chain rule tomulti-variable functions (such as) is rather technical. However, it is simpler to write in the case of functions of the formwhere, and for each
As this case occurs often in the study of functions of a single variable, it is worth describing it separately.
Let, and for eachTo write the chain rule for the composition of functions one needs thepartial derivatives off with respect to itsk arguments. The usual notations for partial derivatives involve names for the arguments of the function. As these arguments are not named in the above formula, it is simpler and clearer to useD-Notation, and to denote by the partial derivative off with respect to itsith argument, and bythe value of this derivative atz.
With this notation, the chain rule is
If the functionf is addition, that is, ifthen and. Thus, the chain rule gives
For multiplicationthe partials are and. Thus,
The case of exponentiationis slightly more complicated, asand, asIt follows that
The simplest way for writing the chain rule in the general case is to use thetotal derivative, which is a linear transformation that captures alldirectional derivatives in a single formula. Consider differentiable functionsf :Rm →Rk andg :Rn →Rm, and a pointa inRn. LetDag denote the total derivative ofg ata andDg(a)f denote the total derivative off atg(a). These two derivatives are linear transformationsRn →Rm andRm →Rk, respectively, so they can be composed. The chain rule for total derivatives is that their composite is the total derivative off ∘g ata:or for short,The higher-dimensional chain rule can be proved using a technique similar to the second proof given above.[8]
Because the total derivative is a linear transformation, the functions appearing in the formula can be rewritten as matrices. The matrix corresponding to a total derivative is called aJacobian matrix, and the composite of two derivatives corresponds to the product of their Jacobian matrices. From this perspective the chain rule therefore says:or for short,
That is, the Jacobian of a composite function is the product of the Jacobians of the composed functions (evaluated at the appropriate points).
The higher-dimensional chain rule is a generalization of the one-dimensional chain rule. Ifk,m, andn are 1, so thatf :R →R andg :R →R, then the Jacobian matrices off andg are1 × 1. Specifically, they are:The Jacobian off ∘g is the product of these1 × 1 matrices, so it isf′(g(a))⋅g′(a), as expected from the one-dimensional chain rule. In the language of linear transformations,Da(g) is the function which scales a vector by a factor ofg′(a) andDg(a)(f) is the function which scales a vector by a factor off′(g(a)). The chain rule says that the composite of these two linear transformations is the linear transformationDa(f ∘g), and therefore it is the function that scales a vector byf′(g(a))⋅g′(a).
Another way of writing the chain rule is used whenf andg are expressed in terms of their components asy =f(u) = (f1(u), …,fk(u)) andu =g(x) = (g1(x), …,gm(x)). In this case, the above rule for Jacobian matrices is usually written as:
The chain rule for total derivatives implies a chain rule for partial derivatives. Recall that when the total derivative exists, the partial derivative in thei-th coordinate direction is found by multiplying the Jacobian matrix by thei-th basis vector. By doing this to the formula above, we find:Since the entries of the Jacobian matrix are partial derivatives, we may simplify the above formula to get:More conceptually, this rule expresses the fact that a change in thexi direction may change all ofg1 throughgm, and any of these changes may affectf.
In the special case wherek = 1, so thatf is a real-valued function, then this formula simplifies even further:This can be rewritten as adot product. Recalling thatu = (g1, …,gm), the partial derivative∂u / ∂xi is also a vector, and the chain rule says that:
Givenu(x,y) =x2 + 2y wherex(r,t) =r sin(t) andy(r,t) = sin2(t), determine the value of∂u / ∂r and∂u / ∂t using the chain rule.[citation needed]and
Faà di Bruno's formula for higher-order derivatives of single-variable functions generalizes to the multivariable case. Ify =f(u) is a function ofu =g(x) as above, then the second derivative off ∘g is:
All extensions of calculus have a chain rule. In most of these, the formula remains the same, though the meaning of that formula may be vastly different.
One generalization is tomanifolds. In this situation, the chain rule represents the fact that the derivative off ∘g is the composite of the derivative off and the derivative ofg. This theorem is an immediate consequence of the higher dimensional chain rule given above, and it has exactly the same formula.
The chain rule is also valid forFréchet derivatives inBanach spaces. The same formula holds as before.[9] This case and the previous one admit a simultaneous generalization toBanach manifolds.
Indifferential algebra, the derivative is interpreted as a morphism of modules ofKähler differentials. Aring homomorphism ofcommutative ringsf :R →S determines a morphism of Kähler differentialsDf : ΩR → ΩS which sends an elementdr tod(f(r)), the exterior differential off(r). The formulaD(f ∘g) =Df ∘Dg holds in this context as well.
The common feature of these examples is that they are expressions of the idea that the derivative is part of afunctor. A functor is an operation on spaces and functions between them. It associates to each space a new space and to each function between two spaces a new function between the corresponding new spaces. In each of the above cases, the functor sends each space to itstangent bundle and it sends each function to its derivative. For example, in the manifold case, the derivative sends aCr-manifold to aCr−1-manifold (its tangent bundle) and aCr-function to its total derivative. There is one requirement for this to be a functor, namely that the derivative of a composite must be the composite of the derivatives. This is exactly the formulaD(f ∘g) =Df ∘Dg.
There are also chain rules instochastic calculus. One of these,Itō's lemma, expresses the composite of an Itō process (or more generally asemimartingale)dXt with a twice-differentiable functionf. In Itō's lemma, the derivative of the composite function depends not only ondXt and the derivative off but also on the second derivative off. The dependence on the second derivative is a consequence of the non-zeroquadratic variation of the stochastic process, which broadly speaking means that the process can move up and down in a very rough way. This variant of the chain rule is not an example of a functor because the two functions being composed are of different types.
