MA 523: Numerical Analysis Instructor: Instructor: Rajendra Kr. Kr. Ray
Credits: 3-1-0-4 Monday: Monday: 8:00 – 8:50 (Lecture) (Lecture) Tuesday: uesday: 14:00 – 14:50 (Lectur (Lecture) e) Wednesda ednesday: y: 11:00 11:00 – 11:50 (Lectur (Lecture) e) Frida Friday: y: 10:00 10:00 – 10:50 10:50 (Tut (Tutori orial) al) 1
Evaluation Pattern
Two Quiz Quizzzes -- 40
Fina Finall Exam Exam --- 50
Tutori utorial al --- 10
Institute requirement:
70% attenda attendance nce
2
References: Texts Books: Atkinson, An Introduction Introduction to Numerical Numerical Analysis, Analysis, 2nd Edition, John Wiley, 2008. K. E. Atkinson, An Analysis, 7th ed., Thomson Learning, 2001. R. L. Burden and J. D. Faires, Numerical Analysis,
References: M. T. Heath, Scientific Computing: An Introductory Survey , McGraw Hill, 2002.
Analysis, McGraw Hill, 1988. S. D. Conte and Carl de Boor, Elementary Numerical Analysis, Rabinowitz, A First Course in Numerical Numerical Analysis, Analysis, Dover Publications, A. Ralston and P. Rabinowitz, A 2001. Bradie, A friendly introduction introduction to Numerical Numerical Analysis, Analysis, Pearson Education, 2007. Brian Bradie, A
3
Approximate Approximate Lecture Distributions: Lecture -1 & 2: Introduction to Numerical Numerical Computing, Significant Significant digits, Sensitivity Sensitivity and conditioning conditioning . Lecture -3 & 4: Approximations Approximations and Errors Errors in Computing , Stability Stability and accuracy Lecture -5 - 9: Roots of Nonlinear Equations Equations (Bisection (Bisection Method, Method, Newton’ Newton’ss methods and its variants , Secant method, Fixed-point iteration method, etc. , order of convergence) Lecture -10 & 16: Existence and uniqueness of interpolating interpolating polynomial, Lagrange Lagrange polynomials, Cubic spline, Least-Square Least-Square approximations. approximations. Lecture -17 & 27: System System of Linear Equations: Direct Solution Solution process (Gauss (Gauss Elimination, Partial Pivoting, etc.), Iterative Solution process (Gauss-Seidel, SOR, QR decomposition, etc.), Lecture -28 & 32: Numerical Differentiation Differentiati on and Integration: Forward, Backward and Central Difference, Newton-Cotes integration formulae, Composite rules, Gaussian quadrature, etc. Lecture -33 & 42: Initial Initia l and Boundary Boundar y Value Problem (IVP & BVP): Taylor series method, Euler methods, Runge-Kutta Runge-Kutta methods, etc., etc., Finite difference difference methods methods and Shooting methods, methods, etc. etc.
Tutorials: Self Learning (SL); Implementation of studied methods, by hand calculations, through computer programming.
4
Unit - 1 : Introduction to Numerical Computing
What is Numerical Computing ? It is an approach for solving complex mathematical problems using only simple arithmetic operations
Numerical Computations: Play an indispensable role in solving real life problems Have been in use even before digital computers appeared The arrival of digital computers has, however, enhanced the speed and accuracy of numerical computations.
Numerical Analysis is the combinations of Mathematical study of the numerical methods with numerical Computing. 5
6
7
8
Characteristics of Numerical Computing Accuracy :
Every method of numerical computing introduces errors Truncation errors Roundoff errors
Rate of Convergence :
Errors introduced into computation, from whatever source, propagate in different ways
Numerical Stability :
Efficiency :
Many numerical methods are based on the idea of an Iterative process
The amount of effort required by both human and computer to implement the method
Mathematical Methods Numerical Computing Computers 9
(1662) (3000 BC.)
(1642)
10
Punched card loom by Jacquard(1801)
11
Difference Engine by Charles Babbage (1822-1834)
12
13
Intel Pentium Processor
14
15
16
Computing Concepts
Processor Chair Mr. Brown
Input
Output
Office Market
DATA
Mr. Brown brought a chair from the market for his office
INFORMATION Instructions
17
18
19
Number Systems and Computer Arithmetic Number System and their Conversions:
Positional Number System
( d p 1d p 2 d p 3 ... d1d 0 .d 1d 2 ... d n )b , Where,
di {0,1, ..., b 1}, i p 1, p 2, ...,1, 0, 1,..., n ;
Decimal value of the number is
D
n
d i bi
i p 1
d p 1
is most Significant digit (MSD)
d n
Is least Significant digit (LSD)
Standard Number Systems
decimal (b=10),
binary (b=2),
octal (b=8),
hexadecimal (b=16).
20
Number system
Base/ radix (b)
Digits {0, 1, … , b-1}
Examples
Binary
2
0, 1
Decimal
10
0, 1, 2, 3, 4, 5, 6, 7, 8, 9
Octal
8
0, 1, 2, 3, 4, 5, 6, 7
Hexadecimal
16
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F
10012 4910 438 4C 216
Relationship between Standard Number Systems : Binary
Decimal
Octal
Hexadecimal
3-bit string
0
0
0
0
000
1
1
1
1
001
10
2
2
2
010
11
3
3
3
011
100
4
4
4
100
(Contd.) 21
Relationship between Standard Number Systems : Binary
Decimal
Octal
Hexadecimal
3-bit string
101
5
5
5
101
110
6
6
6
110
111
7
7
7
111
1000
8
10
8
---
1001
9
11
9
---
1010
10
12
A
---
1011
11
13
B
---
1100
12
14
C
---
1101
13
15
D
---
1110
14
16
E
---
1111
15
17
F
--22
Conversions: Hexadecimal
Decimal:
1 A2C 16 1 163 10 162 2 161 12 160 670010 172810 6 162 C 161 0 160 6C 016 Octal
Decimal:
427.58 4 82 2 81 7 80 5 81 279.62510 427.510 6 82 5 81 3 80 4 81 653.48 Binary
Decimal:
101010 2 1 25 0 2 4 1 23 0 2 2 1 21 0 2 0 4210 18210 1 27 0 26 1 25 1 2 4 0 23 1 2 2
1 21 0 2 0 101101102
23
Fixed-point representation : For an n-bit unsigned binary number, the general form is
d
p 1
d p 2 d p 3 ... d 1 d 0 . d 1 d 2 ... d q
Case 1:
Case 1:
d
If q = 0 and n = p
p 1
d p 2 d p 3 ... d 1 d 0
d
If p = 1 and n = q+1
0
.d 1 d 2 ... d q
2
2
0 i d i 2 . i p 1 10
q d i 2 i . i0 10
X 1100000000000000.2 4914210
Examples: Take n = 16,
Therefore,
2
q i d i 2 , d i {0,1}, p q n. i p 1 10
x 1.1000000000000002 1.510
x X 2
n 1
.
2 n 1
4914210 215
Implicit scale factor 24
Floating-point representation :
x f 10 E .
Basic Representation 31
30
SIGN
1 bit
Examples:
24
23
0
EXPONENT
MANTISSA
7 bits
24 bits
0.00596 10 2 , 0.628 10 5 , 65.7452 E 1, 0.596 E 3
Normalized Representation
0.d1d 2 ..., d n 10 E ,
d i {0,1, 2,..., 9}, d1 0.
Examples:
0.596 100 , 0.628 10 5 , 0.657452 E 1, 0.596 E 3 25
Computer Arithmetic : 25 12 37, Integer Arithmetic
12 25 13, 25 12 300, 25 12 2,
0.9996 103 0.9945 103
0.0050 103 0.5000 105 , 0.735 E 2 0.635 E 2 1.370 E 2 0.137 E 3, 0.200 E 2 0.400 E 2 0.080 E 4 0.800 E 3,
12 25 0.
Floating Point Arithmetic
0.250 102 0.200 103
1.25 101 0.125 102. 26
Errors in Arithmetic : (1 2
24
2
127
) *2
Negative overflow
Negative numbers
126
0
Negative underflow
2
(1 2 ) *2 24
126
Positive underflow
positive numbers
127
Positive overflow
Numbers in the IEEE standard format
Examples :
0.4632 E 99 0.5427 E 99 0.1005E100
(a )
0.9627 E 99 0.9622 E 99 0.5000 E 102
(b )
If we assume that the exponent can have maximum two digits , then the result (a) shows overflow (b) Shows underflow 27
Approximations and Errors in Computing SIGNIFICANT DIGITS :
1 3
22 7
3.1415926535897932384626...
0.333333333333...
3.141 1 3 2 7
0.333
0.285714
2 7
0.285714285714...
four significant digits
three significant digits
six significant digits 28
Notion of Significant Digits : All non-zero digits are significant. All zeros occurring between non-zero digits are significant. Trailing zeros following a decimal point are significant. Zeros between the decimal point and preceding a non-zero digit are not significant. When the decimal point is not written, trailing zeros are
Examples :
Three significant digits: Three significant digits:
significant digits ??
Accuracy and Precision:
7.56, o o
7.5600,
3.50,
not considered.
65.0,
0.0123, 0.00123,
0.756, 756.0, 7.056,
0.230. 1.20.
70.56, 0756.
The number 57.396 is accurate to five significant digits. The number has a precision of 0.001 or 10 3.
29
INHERENT ERRORS : Data Errors
Conversion Errors
Experimental measurements
Representation Errors
Limited accuracy
Computer limitations
0.110 0.00011001 Example:
0.410 0.01100110 Sum 0.01111111
0.25 0.125 0.0625 0.03125 0.015625 0.0078125 0.000390625 0.49609375( 0.5)
Error is equal to
28 0.00390625. 30
Chopping
NUMERICAL ERRORS : (i) Roundoff Errors
Symmetric rounding 42.78
Abs. Err.
Example:
10 E d .
42.7893 42.79
(ii) Truncation Errors
S
a x
e
1 0.2
(0.2) 2
a x
i
2!
(0.2) 3
0.5 10 E d .
n
i
i
i
i 0
0.2
Abs. Err.
i 0
(0.2) 4
3! 4! Truncate
1.22
Truncation. Err.
0.14 102. 31
OTHER ERRORS : (i) Modelling Errors
(ii) Blunders
(iii) Absolute Errors
(iv) Relative Errors
(v) Percentage Errors
Missing Information
Caused due to human inperfection
ea True value Approx. value
er
Abs. Err . Truevalue
er 100 32
Taxonomy of errors
Total Error
Modelling Errors
Inherent Errors
Numerical Errors
Missing information
Data Errors
Measuring method
Blunders
Human imperfection
Conversion Errors
Computing machine
Roundoff Errors
Truncation Errors
Numerical method
33
Error Propagation, Conditioning & Stability Error Propagation: xt yt xa ex ya e y
xa ya ex ey . Total error e x y ex e y
Error bounds
Let
xt xa ex
and yt
ya e y .
xt yt xa ex ya e y
xa ya ex e y . Total error e x y ex e y
e x y e x e y
xt yt xa ex ya ey xa ya ya ex xa ey ex ey e e xa ya ya ex xa ey xa ya xa ya x x y y a a
Total error
e xy
34
xt yt
xa ex ya ey
xa ya ya ex xa ey ex ey ya2 ey2
xa ya ya ex xa ey 2 a
y
xa e x ey y x a a ya ya xa
e x / y Triangle inequality theorem
Errors introduced by the operations
xa ex
Total error
e x / y
e y
ya xa ya e x e y e xy xa ya x y a a xa ya xa ya eo
xt yt xa ya ex ey eo Total error e x y ex e y eo 35
Relative errors: er , x y Addition and Subtraction
Multiplication and division
e x ey xa ya xa xa ya
. er , x
ya xa ya
. er , y
er , xy er , x er , y er , x / y er , x er , y
Example: 4 Estimate the relative error in z = x-y when x = 0.1234 10 and y = 0.1232 10 as stored in a system with four-digit mantissa.
36
4
We Know,
er , z
er , x
e x ey x y
er , y
1 2 1 2
103 0.05% 10 3 0.05%
Then,
e x 0.1234 10 4 0.5 103 0.617 e y 0.1232 10 4 0.5 103 0.616
Therefore,
e x y ex ey 1.233
er , z
1.233 104 0.1234 0.1232
0.6165 61.65% 37
Process graph for u x y z
er , u
er , x y
x x y
x y x y z
er , x
er , x y
y x y
z x y z
er , y r1
u
er, z r2
*
+ z
x
In computing
y
x y z x y z 38
Error Calculation for a Polynomial Function f(x) Let, f ( x ) is differentiable and approximate value x0 of x is given.
According to mean-value theorem
Error in f ( x)
f ( x ) f ( xa ) x xa f '
' e f f ( x ) f ( xa ) ex f
e x max f '
e f ex f ' xa Similarly,
e f ex f x xa , ya ey f y xa , ya 39
Example: Estimate the error in evaluating f ( x, y ) x
Assume that, e x e y
y 4.00. y 2 for x 3.00 and
0.005
f x ( x, y ) 2 x
Therefore,
2
and
f y ( x, y ) 2 y
e f 2 x ex 2 y ey
(2 3.00 2 4.00) 5 103 0.07
40
Error Analysis: Original problem
f ( x ) y
Absolute error
Actually Solved problem f ( x ) y
y y f ( x ) f ( x ) y f ( x )
Exact
x Calculated
y
x
y f ( x )
Exact
x
Results
Input Data
Forward Error: y
( y y)
Backward Error x
( x x)
41
Model
Conditioning and Stability
Sensitivity of problem
Change in Input data
Numerical instability
Sensitivity of numerical method
Induced error
Condition number
Relative error in Relative error in
f ( x) x
er , f er , x
f
.
x
f ( x) x
xf ' ( x) f ( x) 42
Example of ill-conditioned problem: Consider the system of equations
x
Solving,
Let,
m2 m1
y m2 x C 2 ,
;
where m1 and m2 are nearly equal
C1 C2 C 1 m2 m1
y m1
C1 7.00, C2 3.00, m1 2.00, m2 2.01. x
Let,
C1 C2
y m1 x C 1 ,
7.00 3.00 2.01 2.00
400,
y 2.00 400 7.00 807,
C1 7.00, C2 3.00, m1 2.00, m2 2.005. x
7.00 3.00 2.005 2.00
800,
y 2.00 800 7.00 1607. 43
