Roger Crawfis January 30 2009 OSUCIS 541 2 Numerical Differentiation The mathematical definition Can also be thought of as the tangent line x xh January 30 2009 OSUCIS 541 3 Numerical Differentiation ID: 916237
Download Presentation The PPT/PDF document "CSE 541 - Differentiation" is the property of its rightful owner. Permission is granted to download and print the materials on this web site for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Slide1
CSE 541 - Differentiation
Roger Crawfis
Slide2January 30, 2009
OSU/CIS 541
2
Numerical Differentiation
The mathematical definition:
Can also be thought of as the tangent line.
x
x+h
Slide3January 30, 2009
OSU/CIS 541
3
Numerical Differentiation
We can not calculate the limit as
h
goes to zero, so we need to approximate it.
Apply directly for a non-zero h leads to the slope of the secant curve.
x
x+h
Slide4January 30, 2009
OSU/CIS 541
4
Numerical Differentiation
This is called
Forward Differences
and can be derived using Taylor’s Series:
Theoretically speaking
Slide5January 30, 2009
OSU/CIS 541
5
Truncation Errors
Let
f(x)
= a+e, and f(x+h)
= a+f.Then, as h approaches zero, e<<a and f<<a.With limited precision on our computer, our representation of
f(x)
a f(x+h).
We can easily get a random round-off bit as the most significant digit in the subtraction.Dividing by h
, leads to a very wrong answer for f’(x).
Slide6January 30, 2009
OSU/CIS 541
6
Error Tradeoff
Using a smaller step size reduces truncation error.
However, it increases the round-off error.
Trade off/diminishing returns occurs: Always think and test!
Log error
Log step size
Truncation error
Round off error
Total error
Point of
diminishing
returns
Slide7January 30, 2009
OSU/CIS 541
7
Numerical Differentiation
This formula favors (or biases towards) the right-hand side of the curve.
Why not use the left?
x
x+h
x-h
Slide8January 30, 2009
OSU/CIS 541
8
Numerical Differentiation
This leads to the
Backward Differences
formula.
Slide9January 30, 2009
OSU/CIS 541
9
Numerical Differentiation
Can we do better?
Let’s average the two:
This is called the
Central Difference formula.
Forward difference Backward difference
Slide10January 30, 2009
OSU/CIS 541
10
Central Differences
This formula does not
seem
very good.
It does not follow the calculus formula.It takes the slope of the secant with width 2h.The actual point we are interested in is not even evaluated.
x
x+h
x-h
Slide11January 30, 2009
OSU/CIS 541
11
Numerical Differentiation
Is this any better?
Let’s use Taylor’s Series to examine the error:
Slide12January 30, 2009
OSU/CIS 541
12
Central Differences
The central differences formula has much better convergence.
Approaches the derivative as h
2
goes to zero!!
Slide13January 30, 2009
OSU/CIS 541
13
Warning
Still have truncation error problem.
Consider the case of:
Build a table with
smaller values of h.What about largevalues of h
for thisfunction?
Slide14January 30, 2009
OSU/CIS 541
14
Partial Derivatives
Remember: Nothing special about partial derivatives:
Slide15January 30, 2009
OSU/CIS 541
15
Calculating the Gradient
For lab 2, you need to calculate the gradient.
Just use central differences for each partial derivative.
Remember to normalize it (divide by its length).