Consider that we are finding a solution for a nonlinear equation: $f(u) = 0$.
The Newton-Raphson (N-R) iterative scheme is as follows:
Iteration 0: select an initial guess $u_{0}$ for the solution, see Figure 1.
Iteration 1: calculate the value of $f$ at $u_{0}$: $f(u_{0})$, and the value of the first order derivative of $f$ at $u_{0}$: $f'(u_{0})$.
Since $f'(u_{0})$ is actually the slope of the tangent of $f$ at $u_{0}$, we have the following relation (see Figure 2):
$f'(u_{0}) \cdot (u_{1} – u_{0}) = – f(u_{0})$
The above equation can be solved for $u_{1}$.
We can repeat the scheme, see Figure 3.
In general, at iteration $k$, the solution $u_{k}$ can be solved from the equation
$f'(u_{k-1}) \cdot (u_{k} – u_{k-1}) = – f(u_{k-1})$
When will the iterative stop? The scheme stops when th convergence criterion is satisfied. We can choose either
$|f(u_{k})| \leq \epsilon $, or
$| u_{k} – u_{k-1}| \leq \epsilon $, (or $\frac{ | u_{k} – u_{k-1}| } { |u_{k} – u_{0}| } \leq \epsilon $
where $0 < \epsilon << 1$.
The first condition clearly indicates that $u_{k}$ is close to the true solution of $f(u) = 0$, while the second condition just means that the change between two consecutive iterations is quite small (and therefore we stop).
Problem of the N-R scheme:
- We need an initial guess $u_{0}$. If this guess is close to the true solution, the scheme is more likely to converge. If this guess is not close to the true solution, there is high chance that the scheme will not converge. In practice, we usually do not know how “good” is the initial guess!