Approximating a function with polynomials and circles

Branko Ćurgus


Linear approximation

Let $f : I \to \mathbb R$ be a function defined on an open interval $I$, let $a \in I$ and assume that $f$ is differentiable on $I$. By the definition of the derivative \[ f'(a) = \lim_{x\to a} \frac{f(x) - f(a)}{x - a}. \] The last equality can be rewritten as \[ \lim_{x\to a} \frac{f(x) - \bigl( f(a) + f'(a) (x-a) \bigr) }{x - a} = 0. \] The last equality can be interpreted that the error $\bigl|f(x) - \bigl( f(a) + f'(a) (x-a) \bigr) \bigr|$ made in approximating the function $f$ near $a$ by the line $ y = f(a) + f'(a) (x-a)$, is tiny compared to $|x-a|$. Or, in the language of the little-Oh notation: the quantity $f(x) - \bigl( f(a) + f'(a) (x-a) \bigr)$ is $o(x-a)$.
However, the last equation invites further exploration of the quantity \[ f(x) - \bigl( f(a) + f'(a) (x-a) \bigr). \] For example, compare this quantity to $(x-a)^2$ by finding the following limit \[ \lim_{x\to a} \frac{f(x) - \bigl( f(a) + f'(a) (x-a) \bigr) }{(x - a)^2}. \] To asses this limit rewrite it as \[ \lim_{x\to a} \frac{ \frac{f(x) - f(a)}{x-a} - f'(a) }{x - a}. \] The quantity $\displaystyle\frac{f(x) - f(a)}{x-a}$ is the slope of the secant line to the graph of $f$. If $x$ is close to $a$, then this slope is close to both $f'(a)$ and $f'(x)$. Thus, it seems reasonable that replacing $\displaystyle\frac{f(x) - f(a)}{x-a}$ with the average $\displaystyle\frac{f'(x) + f'(a)}{2}$ will not change the above limit: \[ \lim_{x\to a} \frac{ \frac{f(x) - f(a)}{x-a} - f'(a) }{x - a} = \lim_{x\to a} \frac{ \frac{f'(x) + f'(a)}{2} - f'(a) }{x - a} = \lim_{x\to a} \frac{f'(x) - f'(a)}{2(x - a)} \] Assuming that $f$ is twice differentiable at $a$ we have \[ \tag{qa} \lim_{x\to a} \frac{f(x) - \bigl( f(a) + f'(a) (x-a) \bigr) }{(x - a)^2} = \frac{1}{2} f''(a). \] Or, equivalently, \[ \lim_{x\to a} \frac{f(x) - \bigl( f(a) + f'(a) (x-a) + \frac{1}{2} f''(a)(x-a)^2 \bigr) }{(x - a)^2} = 0. \] And the last equality is a good introduction to the next level of approximation.
Higher order approximations

Provided that $f$ is twice differentiable on $I$ and three times differentiable at $a$, this process continues. One can prove using the Mean Value Theorem that \[ \lim_{x\to a} \frac{f(x) - \bigl( f(a) + f'(a) (x-a) + \frac{1}{2} f''(a)(x-a)^2 \bigr) }{(x - a)^3} = \frac{1}{2\cdot 3} f'''(a) = \frac{1}{3!} f'''(a). \] In general, provided that $f$ has all the derivatives on $I$, we have \[ \lim_{x\to a} \frac{f(x) - \sum_{k=0}^n \frac{1}{k!} f^{(k)}(x-a)^k}{(x-a)^{n+1}} = \frac{1}{(n+1)!} f^{(n+1)}(a). \]
Geometric interpretation of the linear approximation

Consider the graph of a differentiable function $y = f(x)$ and a point $\bigl(a,f(a)\bigr)$ on that graph. Which line that passes through the point $\bigl(a,f(a)\bigr)$ is the best approximation for the graph of $y = f(x)$?

Many lines through the point $\bigl(a,f(a)\bigr)$.

This question was answered in the section Linear approximation. The solution was that we choose another point $\bigl(x,f(x)\bigr)$ on the graph of $f$ and consider the limit of the slopes of the secant lines that are determined by the points $\bigl(a,f(a)\bigr)$ and $\bigl(x,f(x)\bigr)$ as $x$ approaches to $a$. This is illustrated by the following animation.

Place the cursor over the image to start the animation.


Osculating circle

Given two non-parallel lines in point-slope form \[ y = m_1 (x- a_1) + b_1, \qquad y = m_2 (x- a_2) + b_2, \] their intersection point is given by \[ \tag{ip} \left( a_1 + \frac{m_2(a_2 - a_1) + b_1 - b_2}{m_2 - m_1}, \ b_1 + m_1 \frac{m_2(a_2 - a_1) + b_1 - b_2}{m_2 - m_1} \right) \] Consider the normal line to the graph of $f$ at the point $(a,f(a))$. That is the line with \[ m_1 = -\frac{1}{f'(a)}, \quad a_1 = a, \quad b_1 = f(a). \] Further consider the symmetry line of the line segment with the endpoints $(a,f(a))$ and $(x,f(x))$. That is the line with \[ m_2 = -\frac{x-a}{f(x)-f(a)}, \quad a_2 = \frac{x+a}{2}, \quad b_2 = \frac{f(x)+f(a)}{2}. \] As we calculated in (ip), the first coordinate of the intersection point of these two lines is \[ a + \frac{-\frac{x-a}{f(x)-f(a)} \frac{x-a}{2} - \frac{f(x)-f(a)}{2}}{-\frac{x-a}{f(x)-f(a)} + \frac{1}{f'(a)}}. \] Simplifying the numerator and the denominator of this double fraction leads to \[ a + \frac{-\frac{(x-a)^2 + (f(x)-f(a))^2}{2(f(x)-f(a))} }{ \ \frac{-f'(a)(x-a) + f(x)-f(a) }{ f'(a) (f(x)-f(a)) } \ }. \] Pulling out $-1/2$ from the numerator and $1/f'(a)$ from the denominator and simplifying the double fraction yields \[ a - \frac{f'(a)}{2} \frac{(x-a)^2 + \bigl( f(x)-f(a) \bigr)^2 }{ f(x) - f(a) - f'(a) (x-a)}. \] Now, dividing by both the numerator and the denominator with $(x-a)^2$ leads to \[ a - \frac{f'(a)}{2} \frac{1 + \left( \frac{ f(x)-f(a) }{x-a} \right)^2 }{ \frac{ f(x)-f(a) - f'(a)(x-a) }{(x-a)^2}}. \] Recall that in (qa) we proved that \[ \lim_{x\to a} \frac{ f(x)-f(a) - f'(a)(x-a) }{(x-a)^2} = \frac{1}{2} f''(a). \] Therefore, based on the algebra of limits, and provided that $f''(a) \neq 0,$ we have \[ \lim_{x\to a} \left( a - \frac{f'(a)}{2} \frac{1 + \left( \frac{ f(x)-f(a) }{x-a} \right)^2 }{ \frac{ f(x)-f(a) - f'(a)(x-a) }{(x-a)^2}} \right) = a - f'(a) \frac{1 + \left(f'(a) \right)^2 }{ f''(a)}. \] Hance \[ a - f'(a) \frac{1 + \left(f'(a) \right)^2 }{ f''(a)} \] is the first coordinate of the center of the osculating circle. To get the second coordinate of the center of the osculating circle we substitute the first coordinate into the equation of the normal line to the graph of $f$ at the point $(a,f(a))$. This gives the center of the osculation circle to the graph of $f$ at the point $(a,f(a))$ to be \[ \left( a - f'(a) \frac{1 + \left(f'(a) \right)^2 }{ f''(a)}, \ f(a) + \frac{1 + \left(f'(a) \right)^2 }{ f''(a)} \right). \] The radius of the osculating circle to the graph of $f$ at the point $(a,f(a))$ is the distance between its center and the point $(a,f(a))$. This distance calculates to: \[ \frac{\left( 1 + \left(f'(a) \right)^2 \right)^{3/2}}{\bigl| f''(a) \bigr|} \] By definition, the reciprocal of the radius of the osculating circle to the graph of $f$ at the point $(a,f(a))$ is called the curvature of the graph of $f$ at the point $(a,f(a))$: \[ \frac{\bigl| f''(a) \bigr|}{\left( 1 + \left(f'(a) \right)^2 \right)^{3/2}}. \]
Geometric illustration of the definition of the osculating circle

Consider the graph of a twice differentiable function $y = f(x)$ and a point $\bigl(a,f(a)\bigr)$ on that graph. Which circle that passes through the point $\bigl(a,f(a)\bigr)$ is the best approximation for the graph of $y = f(x)$? Clearly we need to consider only the circles that touch the tangent line to the graph of $f$ at the point $\bigl(a,f(a)\bigr)$. That is, the circles with the centers on the normal line to the graph of $f$ at the point $\bigl(a,f(a)\bigr)$.

Many circles through the point $\bigl(a,f(a)\bigr)$ that touch the graph of $f$.

Our definition of the osculating circle to the graph of $f$ at the point $\bigl(a,f(a)\bigr)$ is completely analogous to the definition of the tangent line to the graph of $f$ at the point $\bigl(a,f(a)\bigr).$ To see the analogy recall the definition of the slope of the tangent line.
Fix a point $\bigl(a,f(a)\bigr)$ on the graph of $f.$ Next choose another point $\bigl(x,f(x)\bigr)$ on the graph of $f$ and consider the secant line through the points $\bigl(a,f(a)\bigr)$ and $\bigl(x,f(x)\bigr).$ Take the limit of the slopes of the secant lines as $x$ approaches $a.$ That limit is the slope of the tangent line.
Here is the definition of the osculating circle:
Fix a point $\bigl(a,f(a)\bigr)$ on the graph of $f.$ Next choose another point $\bigl(x,f(x)\bigr)$ on the graph of $f$ and consider the secant circle through the points $\bigl(a,f(a)\bigr)$ and $\bigl(x,f(x)\bigr)$ and whose center is on the normal line to the graph of $f$ at the point $\bigl(a,f(a)\bigr).$ Such a circle is uniquely determined and we calculated its center. Take the limit as $x$ approaches to $a$ of such found centers. That limit is the center of the osculating circle to the graph of $f$ at $(a,f(a).$ This is illustrated by the following two animations.

Place the cursor over the image to start the animation.

Place the cursor over the image to start the animation.


Many osculating circles on some familiar graphs

Here are animations of many osculating circles on the graphs of the sine function, the square and the cube function.

Place the cursor over the image to start the animation.

Place the cursor over the image to start the animation.

Place the cursor over the image to start the animation.