Chapter 8

We have discussed methods for solving differential equations using analytical techniques before.

The focus was usually on finding an exact expression for the solution.
There are a lot of important problems in engineering and science that do not apply or are very difficult to use.
The use of numerical approximation methods to get an accurate approximation to the solution of an initial value problem is discussed in this chapter.
The procedures are easy to execute on personal computers and pocket calculators.

We assume that there is a unique solution to the initial value problem in the interval of interest.

The oldest and simplest numerical approximation method is described in Section 2.7.

The method of evaluating Eq.

is used by Euler's method.

The structure of the computer program for Euler's method was pointed out in Section 2.7.

The instructions can be written in any programming language.

Section 2.7 contains some examples of the method.

We don't need numerical methods to solve the initial value problem since the exact solution is known.

The availability of the exact solution makes it easy to determine the accuracy of any numerical procedure that we use.
Throughout the chapter, we will use this problem to demonstrate and compare different numerical methods.
It will be easy to observe the benefits of using more accurate methods when choosing this particular problem.

Some of the results are shown in Table 8.1.1.

Their accuracy is not great.

The numbers are 0.40%, 0.79%, and 1.58%.

It is necessary to use a lot of computation to get good accuracy for this problem.
We will find that it is possible to get comparable or better accuracy with larger step sizes and fewer computational steps when we discuss other numerical approximation methods later in this chapter.

To begin to investigate the errors in using numerical approximations and to suggest ways to construct more accurate algorithms, it is helpful to mention some alternative ways to look at the Euler method.

The derivative is approximated in Eq.

The problem can be written as an equation.

Section 8.2 talks about this.

It is possible to estimate the magnitude of the error in the formula by using a Taylor series with a remainder.

This will be discussed later in this section.

A variation on the formula can be obtained by approximating the derivative.

It will be clear how the method works if we show the first two steps in detail.

The results are shown in Table 8.1.2.

The values given by the backward Euler method are too large for this problem, whereas the values obtained from the Euler method were too small.
There is a question as to why it should be mentioned, since the backward Euler method appears to be no more accurate than the Euler method.
The simplest example of a class of methods known as backward differentiation formulas is very useful for certain types of differential equations.
We will return to this issue later in the chapter.

There are a number of questions that must be answered before the approximate numerical solution can be accepted as satisfactory.

The practical question of how quickly the numerical approximation converges to the solution remains if we assume that the answer is affirmative.

We want to use a step size that is small enough to assure accuracy, but not too small.
An unnecessarily small step size slows down the calculations, makes them more expensive, and in some cases may even cause a loss of accuracy.

Two fundamental sources of error are needed to solve an initial value problem numerically.

We can retain an infinite number of decimal places if our computer is such that we can carry out all computations with complete accuracy.

The second source of error is that we only use a finite number of digits to carry out the computations.

We get from Eq.

The total error is determined by the sum of the truncation and round-off errors.

Useful estimates of the truncation error can be obtained from the numerical procedures discussed in this book.
The local truncation error is what we limit our discussion to.
The round-off error is not random.
Depending on the computer used, the sequence in which the computations are carried out, the method of rounding off, and so on.

While an analysis of round-off error is beyond the scope of this book, it is possible to say more about it than one might think.

Subtracting from the total.

The local truncation error is calculated by applying Eq.

We see from Eq.
immediately.

The expression is given by Eq.

The most difficult thing to do is use any of the Eqs.

The effect that an error at one step will have in succeeding steps is not taken into account in this argument.

The global truncation error of the first order method is proportional to the first power of the step size.

We will use the local truncation error as our main measure of the accuracy of a numerical method because it is more accessible.

It follows from the beginning.

The use of a uniform step size that is smaller than necessary results in more calculations than necessary, more time consumed, and possibly more danger of unacceptable round-off errors.

A method that provides for variations in the step size is called All modern computer codes for solving differential equations have the ability of adjusting the step size as needed.

In the next section, we will return to this question.

The entries are in columns three and four of Table 8.1.1.

The entries are in columns three and four of Table 8.1.2.

The results in parts a and b should be compared with this value.

The development of more efficient methods has been devoted to the improvement of the Euler Method since it requires a very small step size to produce accurate results.

We will discuss some of the methods in the next three sections.

The Euler Formula has been improved.

A better approximate formula can be obtained if the integrand is present.

We get from Eq.

in this way.

This method is a second order method.

To make clear what computations are required, we show a couple of steps.

The improved Euler method yields results that are comparable to, or a bit better than, those generated by the Euler method, with something like one-twelfth of the computing effort.

The improved Euler method is clearly more efficient, yielding substantially better results or requiring less total computing effort.

The improved Euler method can be implemented with a modified computer program.

The possibility of adjusting the step size as a calculation proceeds so as to maintain the local truncation error at a more or less constant level was mentioned in Section 8.1.

At the same time, the goal is to keep some control over the accuracy of the approximation.

The values 1.5 and 1.595 were obtained from the improved Euler method.

The estimated error is 0.095.

It is slightly less than the tolerance.

The actual error is greater than the solution itself.

The local truncation error can be kept constant throughout the entire numerical process by following the same procedure at each step.

Modern adaptive codes for solving differential equations adjust the step size as they go along, although they usually use more accurate formulas than the better ones.
They are able to achieve both efficiency and accuracy by using very small steps.

The exact solution can be found in Section 8.1, if available.

The entries are in columns four and five of Table 8.2.1.

The Runge-Kutta1 class of methods includes the improved Euler methods.

The method originally developed by Runge and Kutta is discussed in this section.

We will follow the method for a short time.
It is two orders of magnitude more accurate than the improved Euler method.
It is easy to use and accurate to handle many problems.
In adaptive Runge-Kutta methods, a provision is made to vary the step size as needed.

6 can be seen as an average slope.

It is not difficult to show that.

We will accept the fact that the local truncation error is in using Eq.

It is not hard to write a computer program to implement this method.

The structure of the program is similar to the one outlined in the book 1Carl David Runge (1856 - 1927), a German mathematician and physicist.

The Runge-Kutta method was used in his paper on the numerical solution of differential equations in 1895.

In 1901, the method was extended to systems of equations.
Kutta was a German mathematician and aerodynamicist who was well known for his contributions to classical airfoil theory.

Chapter 3 of the book by Henrici is listed in the references.

The local truncation error in the Runge-Kutta formula is consistent with 5.

2 is correct.

While this error may be acceptable for some purposes, it is more than 135 times the error yielded by the Runge-Kutta method.

A more accurate algorithm is more efficient because it produces better results with similar effort or less effort.

If available, compare the results with those obtained using other methods.

If you want to confirm the results, execute the indicated computations.

There are two types of multistep methods, Adams3 methods and backward differentiation formulas.

Depending on the number of preceding data points used, one can achieve various levels of accuracy within each type.

John Couch Adams, an English astronomer, was the codiscoverer of the planet Neptune.

Adams' procedure for numerical integration of differential equations appeared in a book with Francis Bashforth on capillary action.

The second order Adams-Bashforth formula has Equation 5.

The procedure outlined above can be used to get more accurate Adams formulas.

The Adams-Moulton4 formulas are a variation of the Adams-Bashforth formulas.

Let's look at the second order case again.

An astronomer and administrator of science, Forest Ray Moulton was an American.

During World War I, he devised improvements to the Adams formula.

The first order Adams-Moulton formula is similar to the first order Adams-Bashforth formula.

An approximating polynomial of higher degree can be used to get more accurate higher order formulas.

For the fourth order formulas, the proportionality constant for the Adams-Moulton formula is less than for the Adams-Bashforth formula.

The answer depends on whether using the more accurate formula can increase the step size and reduce the number of steps enough to compensate for the additional computations required at each step.

This method is used in the next example.

The fourth order Adams-Bashforth formula, the fourth order Adams- Moulton formula, and the predictor-corrector method are used.

The Adams-Bashforth method is the simplest and fastest method, since it only involves evaluation of a single explicit formula.

It's the least accurate.
The method is still explicit when using the Adams-Moulton formula as a corrector.

The Adams-Moulton method yields the best result, with an error that is about 1/40 as large as the predictor- corrector method.

The Adams-Moulton method requires an equation to be solved at each step.
This equation is linear, so the solution is quickly found, but in other problems this part of the procedure may take more time.

The Runge-Kutta method is comparable in accuracy to the predictor-corrector method.

Backward differentiation formulas of any order can be obtained by using higher order polynomials and more data points.

The Eq is being used.

An error is found in 7942260.

This is somewhat better than the result using the Adams- Bashforth method, but not as good as the result using the predictor-corrector method, and not nearly as good as the result using the Adams-Moulton method.

There are a number of factors that must be considered in a comparison between one-step and multistep methods.

If Runge-Kutta can use fewer steps, the difference in speed will be eliminated.

Difficulty in solving the implicit equation at each step is taken into account by the Adams-Moulton and backward differentiation formulas.
Errors in earlier steps can feed back into later calculations with unfavorable consequences.
It is possible to approximate the solution at points between the mesh points with the underlying polynomial approximations.
Multistep methods have become popular because it is easy to estimate the error at each step and to adjust the order or the step size to control it.
The books listed at the end of the chapter are authoritative sources for further discussion of these questions.

The specified method is used 5 times.

If available, compare the results of the various methods with each other.

The second order backward differentiation formula is given by Eq.

There are some other difficulties that can arise in this section.

Some of the points we want to make are difficult to treat in detail, so we will show them by examples.

Determine the optimum step size.

The data shown in Table 8.5.1 is obtained by keeping only four digits.

There are quantities in the third and fifth columns.

The difference between the calculated values and the actual value of the solution can be seen in the fourth and sixth columns.

The global trun cation error is larger than the round-off error.

The error is reduced proportionally when the step size is reduced.
The round-off error has been further reduced, but less than proportionally.

If too many steps are required in a calculation, then eventually round-off error is likely to accumulate to the point that it seriously degrades the accuracy of the procedure.

For many problems, the fourth order methods discussed in Sections 8.3 and 8.4 will produce good results with a number of steps far less important than the level at which round-off error becomes important.

Round-off error is important for some problems.

The choice of method may be crucial.
Modern codes give a means of adjusting the step size as they go along, using a larger step size wherever possible, and a very small step size only where necessary.

Different numerical procedures are used.

The Runge-Kutta method is a more accurate numerical procedure because of the large differences among the computed values.

It's quite different from the ones obtained using the Euler method.

Table 8.5.2 contains interesting information.

This example shows the kind of information that can be obtained by combining analytical and numerical work.

There is a possibility that small errors that are introduced in the course of a mathematical procedure may die out as the procedure continues.

Small errors can cause instability if they increase.
The solution of an initial value problem is stable if initially nearby solutions tend to approach it, and unstable if they depart from it.
In an stable problem the graphs of solutions will come together, while in an unstable problem they will separate.

If we are trying to solve an initial value problem numerically, the best we can hope for is that the numerical approximation mimics the actual solution.

We can't make an unstable problem into a stable one by numerically solving it.

It is possible that a numerical procedure will introduce instabilities that were not part of the original problem, and this can cause trouble in approximating the solution.

We can compare the solution of Eq.

The Eqs are compared.

This confirms our conclusion in Chapter 2.

This quantity is always non negative and never greater than 1.

It is large.

The most popular formulas for solving stiff problems are described in Section 8.4.
In trying to approximate the solution of a stiff problem, there are instabilities that can occur.

The Runge-Kutta and A Stiff ward methods are used.

The solution varies rapidly if the exponential term is significant.

The width of the boundary layer is small.

If we plan to approximate the solution numerically, we might expect a small step size in the boundary layer.

There is no restriction for the backward Euler method.

The results are shown in columns 3 and 4 of the table.

The results in column 7 of the table show 1 using the backward Euler method.

The situation is not improved by using a more accurate method.

The results in columns 5 and 6 are shown in Table 8.5.3.

A smaller step size is needed in the boundary layer to get an accurate approximation.

In Section 8.6, the generalization of numerical techniques for the first order equations to higher order equations or to systems of equations is discussed, but that is not needed for the present discussion.

If only a fixed number of digits are retained, these two functions look the same.

The results are in Table 8.5.4.

If no further errors are introduced in the intervening calculations, the value is 5.

The results show that this is what happens.

The behavior shown in this example is typical of unstable problems.

One can track a solution accurately for a while, and the interval can be extended by using smaller step sizes or more accurate methods, but eventually the instability in the problem itself takes over and leads to large errors.

Several numerical methods have been introduced for approximating the solution of an initial value problem.

We tried to emphasize some important ideas while maintaining a reasonable level of complexity.
For one thing, we have always used a uniform step size, whereas production codes currently in use provide for varying the step size as the calculation proceeds.

There are a number of factors that must be taken into account when choosing step sizes.

It's true that one is accuracy, but too large a step size leads to an inaccurate result.

The step size at each step must be in line with the error tolerance prescribed in advance.

The method must be stable if the step size is chosen.
Small errors will grow and render the results worthless.
For implicit methods, an equation must be solved at each step and the method used to solve it may impose additional restrictions on the step size.

The amount of time required to execute each step must be considered when choosing a method.

An implicit method, such as the Adams-Moulton method, requires more calculations for each step, but if its accuracy and stability allow a larger step size, then this may more than compensate for the additional calculations.
Stable formulas of moderate order are indicated for stiff problems for which stability is the controlling factor.

As the calculation proceeds, some current production codes allow the order of the method to be varied, as well as the step size.

The order and size of the steps are chosen to satisfy the prescribed error tolerance.
Adams methods can be used up to twelve and five.
There is a lack of stability in higher order backward differentiation formulas.

The initial value problem can be considered again.

Determine how small a step is using the Euler method.

The first order differential equation has initial value problems associated with it.

The methods can be applied to the first order equations.
It is sufficient to deal with systems of first order equations alone since a higher order equation can always be reduced to a system of first order equations.

The Runge-Kutta method can be extended to a system.

The equations are valid in any number of dimensions.

The first thing we should do is use the Euler method.

The following values were obtained.

Substitute these values in Eq.

This example shows the great gains in accuracy that can be achieved by using a more accurate approximation method.

The error in the Runge-Kutta method is 200 times less than the error in the Euler method.

You can compare the results obtained by different methods.

There are many books that deal with numerical analysis in general and the numerical solution of ordinary differential equations.

There are chapters on differential equations in many books.