7.9 Nonhomogeneous Linear Systems
Chapter 7. Systems of First Order Linear Equations
7.2 Review of Matrices
For both theoretical and computational reasons it is advisable to bring to bear some of the results of matrix theory2 on the initial value problem for a system of linear differential equations. For reference purposes this section and the next are devoted to a brief summary of the facts that will be needed later. More details can be found in any elementary book on linear algebra. We assume, however, that you are familiar with determinants and how to evaluate them.
We designate matrices by boldfaced capitals A, B, C, . . . , occasionally using bold faced Greek capitals ⌽, ⌿, . . . . A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns, that is,
a
a · · · a
11
12
1n
aa · · · a
21
22
2n
A =
..
.
.
.
(1) .
..
..
a
a · · · a
m1
m2
mn
We speak of A as an m × n matrix. Although later in the chapter we will often assume that the elements of certain matrices are real numbers, in this section we assume that the elements of matrices may be complex numbers. The element lying in the i th row and j th column is designated by a , the first subscript identifying its row and the i j second its column. Sometimes the notation (a ) is used to denote the matrix whose i j generic element is a .
i j
Associated with each matrix A is the matrix AT , known as the transpose of A, and obtained from A by interchanging the rows and columns of A. Thus, if A = (a ), i j then AT = (a ). Also, we will denote by a the complex conjugate of a , and by Aj i
i j
i j the matrix obtained from A by replacing each element a by its conjugate a . The i j
i j matrix A is called the conjugate of A. It will also be necessary to consider the transpose
T of the conjugate matrix A . This matrix is called the A and will be denoted by A∗.
For example, let
3
2 − i
A = .
4 + 3i −5 + 2i
Then
3
4 + 3i
3
2 + iAT = ,A = ,
2 − i −5 + 2i 4 − 3i −5 − 2i
3
4 − 3i
A∗ = .
2 + i −5 − 2i 2The properties of matrices were first extensively explored in 1858 in a paper by the English algebraist Arthur Cayley (1821–1895), although the word matrix was introduced by his good friend James Sylvester (1814 –1897) in 1850. Cayley did some of his best mathematical work while practicing law from 1849 to 1863; he then became professor of mathematics at Cambridge, a position he held for the rest of his life. After Cayley’s groundbreaking work, the development of matrix theory proceeded rapidly, with significant contributions by Charles Hermite, Georg Frobenius, and Camille Jordan, among others.
7.2
Review of Matrices
We are particularly interested in two somewhat special kinds of matrices: square matrices, which have the same number of rows and columns, that is, m = n; and vectors (or column vectors), which can be thought of as n × 1 matrices, or matrices having only one column. Square matrices having n rows and n columns are said to be of order n. We denote (column) vectors by boldfaced lowercase letters, x, y, , , . . . . The transpose xT of an n × 1 column vector is a 1 × n row vector, that is, the matrix consisting of one row whose elements are the same as the elements in the corresponding positions of x.
Properties of Matrices.
1. Equality. Two m × n matrices A and B are said to be equal if all corresponding elements are equal, that is, if a = b for each i and j.
i j
i j
2. Zero. The symbol 0 will be used to denote the matrix (or vector), each of whose elements is zero.
3. Addition. The sum of two m × n matrices A and B is defined as the matrix obtained by adding corresponding elements: A + B = (a ) + (b ) = (a + b ).
(2)
i j
i j
i j
i j
With this definition it follows that matrix addition is commutative and associative, so that A + B = B + A,A + (B + C) = (A + B) + C.
(3) 4. Multiplication by a Number. The product of a matrix A by a complex number α is defined as follows: αA = α(a ) = (αa ).
(4)
i j
i j
The distributive laws α(A + B) = αA + αB,(α + β)A = αA + βA (5)
are satisfied for this type of multiplication. In particular, the negative of A, denoted by −A, is defined by −A = (−1)A.
(6) 5. Subtraction. The difference A − B of two m × n matrices is defined by A − B = A + (−B).
(7)
Thus A − B = (a ) − (b ) = (a − b ),
(8)
i j
i j
i j
i j which is similar to Eq. (2).
6. Multiplication. The product AB of two matrices is defined whenever the number of columns of the first factor is the same as the number of rows in the second. If A and B are m × n and n × r matrices, respectively, then the product C = AB is an m × r
matrix. The element in the i th row and j th column of C is found by multiplying each element of the i th row of A by the corresponding element of the j th column of B and then adding the resulting products. Symbolically,
n
c = a b .
(9)
i ji k k j
k=1
Chapter 7. Systems of First Order Linear Equations
By direct calculation it can be shown that matrix multiplication satisfies the associative law (AB)C = A(BC)
(10)
and the distributive law A(B + C) = AB + AC.
(11)
However, in general, matrix multiplication is not commutative. For both products AB
and BA to exist and to be of the same size, it is necessary that A and B be square matrices of the same order. Even in that case the two products are usually unequal, so that, in general AB = BA.
(12)
To illustrate the multiplication of matrices, and also the fact that matrix multiplication E X A M P L E is not necessarily commutative, consider the matrices
1
1
−2
1
2
1
−1 A = 0
2
−1 ,
B = 1
−1
0 .
2
1
1
2
−1
1
From the definition of multiplication given in Eq. (9) we have
2 − 2 + 2 1 + 2 − 1 −1 + 0 + 1 AB = 0 + 2 − 2 0 − 2 + 1 0 + 0 − 1 4 + 1 + 2 2 − 1 − 1 −2 + 0 + 1
2
2
0
= 0 −1 −1 .
7
0
−1
Similarly, we find that
0
−3
0
BA = 1
−4
2 .
4
−5
4
Clearly, AB = BA.
7. Multiplication of Vectors. Matrix multiplication also applies as a special case if the matrices A and B are 1 × n and n × 1 row and column vectors, respectively.
Denoting these vectors by xT and y we have
n
xT y = x y .
(13)
i i
i =1
This is the extension to n dimensions of the familiar dot product from physics and calculus. The result of Eq. (13) is a (complex) number, and it follows directly from Eq. (13) that xT y = yT x,xT (y + z) = xT y + xT z,(αx)T y = α(xT y) = xT (αy).
(14)
7.2
Review of Matrices
There is another vector product that is also defined for any two vectors having the same number of components. This product, denoted by (x, y), is called the scalar or and is defined by
n
(x, y) = x y ,
(15)
i
i
i =1
The scalar product is also a (complex) number, and by comparing Eqs. (13) and (15) we see that (x, y) = xT y.
(16)
Thus, if all of the elements of y are real, then the two products (13) and (15) are identical. From Eq. (15) it follows that (x, y) = (y, x),(x, y + z) = (x, y) + (x, z), (17)
(αx, y) = α(x, y),(x, αy) = α(x, y).
Note that even if the vector x has elements with nonzero imaginary parts, the scalar product of x with itself yields a nonnegative real number,
n
n
(x, x) = x x = |x |2.
(18)
i
i
i
i =1 i =1
The nonnegative quantity (x, x)1/2, often denoted by x, is called the length, or magnitude, of x. If (x, y) = 0, then the two vectors x and y are said to be For example, the unit vectors i, j, k of three-dimensional vector geometry form an orthogonal set. On the other hand, if some of the elements of x are not real, then the matrix product
n
xT x =
x2
(19)
i
i =1 may not be a real number.
For example, let
i
2 − ix = −2 ,y = i .
1 + i
3
Then xT y = (i)(2 − i) + (−2)(i) + (1 + i)(3) = 4 + 3i,(x, y) = (i)(2 + i) + (−2)(−i) + (1 + i)(3) = 2 + 7i,xT x = (i)2 + (−2)2 + (1 + i)2 = 3 + 2i,(x, x) = (i)(−i) + (−2)(−2) + (1 + i)(1 − i) = 7.
8. Identity. The multiplicative identity, or simply the identity matrix I, is given by
1
0
· · · 0 0 1 · · · 0
I =
..
.
. .
(20) .
..
..
0
0
· · · 1
Chapter 7. Systems of First Order Linear Equations
From the definition of matrix multiplication we have AI = IA = A (21)
for any (square) matrix A. Hence the commutative law does hold for square matrices if one of the matrices is the identity.
9. Inverse. The matrix A is said to be nonsingular or invertible if there is another matrix B such that AB = I and BA = I, where I is the identity. If there is such a B, it can be shown that there is only one. It is called the multiplicative inverse, or simply the inverse, of A, and we write B = A−1. Then AA−1 = A−1A = I.
(22)
Matrices that do not have an inverse are called singular or noninvertible.
There are various ways to compute A−1 from A, assuming that it exists. One way involves the use of determinants. Associated with each element a of a given matrix is i j the minor M , which is the determinant of the matrix obtained by deleting the i th row i j and j th column of the original matrix, that is, the row and column containing a . Also i j associated with each element a is the cofactor C defined by the equation i j
i j
C
= (−1)i+j M .
(23)
i j
i j If B = A−1, then it can be shown that the general element b is given by i j
Cji
b =
.
(24)
i j
det A While Eq. (24) is not an efficient way3 to calculate A−1, it does suggest a condition that A must satisfy for it to have an inverse. In fact, the condition is both necessary and sufficient: A is nonsingular if and only if det A = 0. If det A = 0, then A is singular.
Another and usually better way to compute A−1 is by means of elementary row operations. There are three such operations:
1.
Interchange of two rows.
2.
Multiplication of a row by a nonzero scalar.
3.
Addition of any multiple of one row to another row.
Any nonsingular matrix A can be transformed into the identity I by a systematic sequence of these operations. It is possible to show that if the same sequence of operations is then performed on I, it is transformed into A−1. The transformation of a matrix by a sequence of elementary row operations is referred to as row reduction or Gaussian elimination. The following example illustrates the calculation of an inverse matrix in this way.
3For large n the number of multiplications required to evaluate A−1 by Eq. (24) is proportional to n!. If one uses more efficient methods, such as the row reduction procedure described later, the number of multiplications is proportional only to n3. Even for small values of n (such as n = 4), determinants are not an economical tool in calculating inverses, and row reduction methods are preferred.
7.2
Review of Matrices
Find the inverse of
E X A M P L E
1
−1 −1
2
A = 3
−1
2 .
2
2
3
The matrix A can be transformed into I by the following sequence of operations.
The result of each step appears below the statement.
(a) Obtain zeros in the off-diagonal positions in the first column by adding (−3) times the first row to the second row and adding (−2) times the first row to the third row.
1
−1 −1
0
2
5
0
4
5
(b) Obtain a 1 in the diagonal position in the second column by multiplying the second row by 1 .
2
1
−1 −1
0
1
5
2
0
4
5
(c) Obtain zeros in the off-diagonal positions in the second column by adding the second row to the first row and adding (−4) times the second row to the third row.
1
0
3
2
0
1
5
2
0
0
−5
(d) Obtain a 1 in the diagonal position in the third column by multiplying the third row by (− 1 ).
5
1
0
3
2
0
1
5
2
0
0
1
(e) Obtain zeros in the off-diagonal positions in the third column by adding (− 3 )
2
times the third row to the first row and adding (− 5 ) times the third row to the
2
second row.
1
0
0
0 1 0
0
0
1
If we perform the same sequence of operations in the same order on I, we obtain the following sequence of matrices:
1
0
0
1
0
0
1
0
0
0
1
0 , −3 1 0 ,
−3
1
0 ,
2
2
0
0
1
−2 0 1 −2 0 1
Chapter 7. Systems of First Order Linear Equations
−1
1
0
−1
1
0
7
− 1
3
2
2
2
2
10
10
10
−
3
1
0 , −3
1
0 , 1 −1 1 .
2
2
2
2
2
2
2
4
−2
1
−4
2
−1
−4
2
−1
5
5
5
5
5
5
The last of these matrices is A−1, a result that can be verified by direct multiplication with the original matrix A.
This example is made slightly simpler by the fact that the original matrix A had a 1 in the upper left corner (a = 1). If this is not the case, then the first step is to produce
11
a 1 there by multiplying the first row by 1/a , as long as a = 0. If a = 0, then the
11
11
11
first row must be interchanged with some other row to bring a nonzero element into the upper left position before proceeding.
Matrix Functions.
We sometimes need to consider vectors or matrices whose ele ments are functions of a real variable t. We write
x (t)a (t) · · · a (t) 1 11
1n
x(t) = .. .
.
. ,
A(t) = .. .. ,
(25)
x (t)a (t) · · · a (t)
n
n1
nn
respectively.
The matrix A(t) is said to be continuous at t = t or on an interval α < t < β if
0
each element of A is a continuous function at the given point or on the given interval.
Similarly, A(t) is said to be differentiable if each of its elements is differentiable, and its derivative dA/dt is defined by
dA
da
=
i j ;
(26)
dt
dt
that is, each element of dA/dt is the derivative of the corresponding element of A. In the same way the integral of a matrix function is defined as
b
b
A(t) dt = a (t) dt .
(27)
i ja
a For example, if
sin tt
A(t) = ,
1
cos t then
π
cos t
1
2
π2/2
A(t) = ,
A(t) dt = .
0
− sin t
π
0
0
Many of the rules of elementary calculus extend easily to matrix functions; in particular,
d (
dA
CA) = C,
where C is a constant matrix;
(28)
dt
dt
d (A + B) = dA + dB;
(29)
dt
dt
dt
d (
dB
AB) = A + dAB.
(30)
dt
dt
dt
7.2
Review of Matrices In Eqs. (28) and (30) care must be taken in each term to avoid interchanging the order of multiplication. The definitions expressed by Eqs. (26) and (27) also apply as special cases to vectors.
PROBLEMS
1
−2
0
4
−2
3
2
5
0, find
−2
1
3
6
1
2
i
3
, find
2
−2
1
2
1
2
3
0
2
−1
1
−2
1
0
, find
3
2
−1
2
1
−1
−1
3
1
2
1
1
0
2
1
−2
0
2
1
−1
2
1
0
2
3
2
2, verify that
−2
0
3
1
0
2
0
1
−1
7. Prove each of the following laws of matrix algebra:
2
2 , find
Chapter 7. Systems of First Order Linear Equations
1 − 2i
2
9. If x =
i and y = 3 − i , show that
2
1 + 2i (a) xT y = yT x (b) (x, y) = (y, x) In each of Problems 10 through 19 either compute the inverse of the given matrix, or else show that it is singular.
1
4
3
−1
10.
−
11.
2
3
6
2
1
2
3
1
1
−1
12. 2
4
5
13. 2
−1
1
3
5
6
1
1
2
1
2
1
2
1
0
14. −2
1
8
15. 0
2
1
1
−2 −7
0
0
2
1
−1 −1
2
3
1
16. 2
1
0
17. −1
2
1
3
−2
1
4
−1 −1
1
0
0
−1
1
−1
2
0
0 −1
1
0
−1
2
−4
2
18. −
19.
1
0
1
0
1
0
1
3
0
1
−1
1
−2
2
0
−1
et
1
(d)
0
In each of Problems 22 through 24 verify that the given vector satisfies the given differential equation.
3
−2
4
2
−2
2
2
−1
1
1
1
23. x = x + et ,
x = et + 2
tet
3
−2
−1
0
1
1
1
1
6
0
24. x = 2
1
−1 x,x = −8 e−t + 2 1 e2t
0
−1
1
−4
−1
7.3Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors In each of Problems 25 and 26 verify that the given matrix satisfies the given differential equation.
1
1
25. ⌿ =
4
−2
1
−1
4
et
e−2t
e3t
26. ⌿ = 3
2
−1 ⌿, ⌿(t) = −4et −e−2t 2e3t
2
1
−1
−et
−e−2t
e3t
7.3 Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors
In this section we review some results from linear algebra that are important for the solution of systems of linear differential equations. Some of these results are easily
ODE
proved and others are not; since we are interested simply in summarizing some useful information in compact form, we give no indication of proofs in either case. All the results in this section depend on some basic facts about the solution of systems of linear algebraic equations.
Systems of Linear Algebraic Equations.
A set of n simultaneous linear algebraic equations in n variables, a x + a x + · · · + a x = b , 11 1
12 2 1n n
1
...
(1)
a x + a x + · · · + a x = b ,n1 1
n2 2 nn n
n
can be written as Ax = b, (2)
where the n × n matrix A and the vector b are given, and the components of x are to be determined. If b = 0, the system is said to be otherwise, it is If the coefficient matrix A is nonsingular, that is, if det A is not zero, then there is a unique solution of the system (2). Since A is nonsingular, A−1 exists, and the solution can be found by multiplying each side of Eq. (2) on the left by A−1; thus x = A−1b.
(3)
In particular, the homogeneous problem Ax = 0, corresponding to b = 0 in Eq. (2), has only the trivial solution x = 0.
On the other hand, if A is singular, that is, if det A is zero, then solutions of Eq. (2)
either do not exist, or do exist but are not unique. Since A is singular, A−1 does not exist, so Eq. (3) is no longer valid. The homogeneous system Ax = 0 (4)
Chapter 7. Systems of First Order Linear Equations has (infinitely many) nonzero solutions in addition to the trivial solution. The situation for the nonhomogeneous system (2) is more complicated. This system has no solution unless the vector b satisfies a certain further condition. This condition is that (b, y) = 0, (5)
for all vectors y satisfying A∗y = 0, where A∗ is the adjoint of A. If condition (5) is met, then the system (2) has (infinitely many) solutions. Each of these solutions has the form x = x(0) + , (6)
where x(0) is a particular solution of Eq. (2), and is any solution of the homogeneous system (4). Note the resemblance between Eq. (6) and the solution of a nonhomogeneous linear differential equation. The proofs of some of the preceding statements are outlined in Problems 25 through 29.
The results in the preceding paragraph are important as a means of classifying the solutions of linear systems. However, for solving particular systems it is generally best to use row reduction to transform the system into a much simpler one from which the solution(s), if there are any, can be written down easily. To do this efficiently we can form the augmented matrix
a · · · a | b 11
1n
1 A | b = .. .
.
..
| ...
(7)
a · · · a | b
n1
nn
n by adjoining the vector b to the coefficient matrix A as an additional column. The dashed line replaces the equals sign and is said to partition the augmented matrix.
We now perform row operations on the augmented matrix so as to transform A into a triangular matrix, that is, a matrix whose elements below the main diagonal are all zero. Once this is done, it is easy to see whether the system has solutions, and to find them if it does. Observe that elementary row operations on the augmented matrix (7) correspond to legitimate operations on the equations in the system (1). The following examples illustrate the process.
Solve the system of equations E X A M P L E
1
x − 2x + 3x = 7, 1
2
3
−x + x − 2x = −5,
(8)
1
2
3
2x − x − x = 4.
1
2
3
The augmented matrix for the system (8) is
1
−2
3
|
7
−1
1
−2 | −5
(9)
2
−1 −1 |
4
We now perform row operations on the matrix (9) with a view to introducing zeros in the lower left part of the matrix. Each step is described and the result recorded below.
7.3Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors (a) Add the first row to the second row and add (−2) times the first row to the third row.
1
−2
3
|
7
0 −1
1
|
2
0
3
−7 | −10
(b) Multiply the second row by −1.
1
−2
3
|
7
0
1
−1 |
−2
0
3
−7 | −10
(c) Add (−3) times the second row to the third row.
1
−2
3
|
7
0
1
−1 | −2
0
0
−4 | −4
(d) Divide the third row by −4.
1
−2
3
|
7
0
1
−1 | −2
0
0
1
|
1
The matrix obtained in this manner corresponds to the system of equations x − 2x + 3x = 7, 1
2
3
x − x = −2,
(10)
2
3
x = 1, 3
which is equivalent to the original system (8). Note that the coefficients in Eqs. (10)
form a triangular matrix. From the last of Eqs. (10) we have x = 1, from the second
3
equation x = −2 + x = −1, and from the first equation x = 7 + 2x − 3x = 2.
2
3
1
2
3
Thus we obtain
2
x = −1 , 1
which is the solution of the given system (8). Incidentally, since the solution is unique, we conclude that the coefficient matrix is nonsingular.
Discuss solutions of the system E X A M P L E
2
x − 2x + 3x = b ,
1
2
3
1
−x + x − 2x = b ,
(11)
1
2
3
2
2x − x + 3x = b
1
2
3
3
for various values of b , b , and b .
1
2
3
Chapter 7. Systems of First Order Linear Equations
Observe that the coefficients in the system (11) are the same as those in the system (8) except for the coefficient of x in the third equation. The augmented matrix for the
3
system (11) is
1
−2
3
| b
1
−1
1
−2 | b .
(12)
2
2
−1
3
| b3
By performing steps (a), (b), and (c) as in Example 1 we transform the matrix (12) into
1
−2
3
|
b
1
0
1
−1 | −b − b .
(13)
1
2
0
0
0
| b + 3b + b
1
2
3
The equation corresponding to the third row of the matrix (13) is b + 3b + b = 0; (14)
1
2
3
thus the system (11) has no solution unless the condition (14) is satisfied by b , b , and
1
2
b . It is possible to show that this condition is just Eq. (5) for the system (11).
3 Let us now assume that b = 2, b = 1, and b = −5, in which case Eq. (14) is
1
2
3
satisfied. Then the first two rows of the matrix (13) correspond to the equations x − 2x + 3x = 2, 1
2
3
(15)
x − x = −3.
2
3
To solve the system (15) we can choose one of the unknowns arbitrarily and then solve for the other two. Letting x = α, where α is arbitrary, it then follows that
3
x = α − 3, 2
x = 2(α − 3) − 3α + 2 = −α − 4.
1
If we write the solution in vector notation, we have
−α − 4
−1
−4
x = α − 3 = α 1 + −3 .
(16)
α
1
0
It is easy to verify that the second term on the right side of Eq. (16) is a solution of the nonhomogeneous system (11), while the first term is the most general solution of the homogeneous system corresponding to (11).
Row reduction is also useful in solving homogeneous systems, and systems in which the number of equations is different from the number of unknowns.
Linear Independence.
A set of k vectors x(1), . . . , x(k) is said to be linearly depen- dent if there exists a set of (complex) numbers c , . . . , c , at least one of which is
1
k
nonzero, such that c x(1) + · · · + c x(k) = 0.
(17)
1
k
In other words x(1), . . . , x(k) are linearly dependent if there is a linear relation among them. On the other hand, if the only set c , . . . , c for which Eq. (17) is satisfied is
1
k
c = c = · · · = c = 0, then x(1), . . . , x(k) are said to b
1
2
k
( Consider now a set of n vectors, each of which has n components. Let x = x j) be i j
i
the i th component of the vector x( j), and let X = (x ). Then Eq. (17) can be written as i j
(
(x 1)c + · · · + x n)cx c + · · · + x c 1 1
1
n
11 1 1n n .
.
.
.
.
.
.. = .. .. = Xc = 0.
(18) x(1)c + · · · + x(n)cx c + · · · + x c
n
1
n
nn1 1 nn n If det X = 0, then the only solution of Eq. (18) is c = 0, but if det X = 0, there are nonzero solutions. Thus the set of vectors x(1), . . . , x(n) is linearly independent if and only if det X = 0.
Determine whether the vectors E X A M P L E
3
1
2
−4 x(1) = 2 ,x(2) = 1 ,x(3) =
1
(19)
−1
3
−11
are linearly independent or linearly dependent. If linearly dependent, find a linear relation among them.
To determine whether x(1), x(2), and x(3) are linearly dependent we compute det(x ), i j whose columns are the components of x(1), x(2), and x(3), respectively. Thus
1
2
−4
det(x ) = 2
1
1 ,i j
−1
3
−11
and an elementary calculation shows that it is zero. Thus x(1), x(2), and x(3) are linearly dependent, and there are constants c , c , and c such that
1
2
3
c x(1) + c x(2) + c x(3) = 0.
(20)
1
2
3
Equation (20) can also be written in the form
1
2
−4
c
0
1
2
1
1 c = 0 ,
(21)
2
−1
3
−11
c
0
3
and solved by means of elementary row operations starting from the augmented matrix
1
2
−4 | 0 2
1
1
| 0 .
(22)
−1
3
−11 | 0
We proceed as in Examples 1 and 2.
(a) Add (−2) times the first row to the second row, and add the first row to the third row.
1
2
−4 | 0
0 −3
9
| 0
0
5
−15 | 0
Chapter 7. Systems of First Order Linear Equations (b) Divide the second row by −3; then add (−5) times the second row to the third row.
1
2
−4 | 0
0
1
−3 | 0
0
0
0
| 0
Thus we obtain the equivalent system c + 2c − 4c = 0, 1
2
3
(23) c − 3c = 0.
2
3
From the second of Eqs. (23) we have c = 3c , and from the first we obtain c =
2
3
1
4c − 2c = −2c . Thus we have solved for c and c in terms of c , with the latter
3
2
3
1
2
3
remaining arbitrary. If we choose c = −1 for convenience, then c = 2 and c = −3.
3
1
2
In this case the desired relation (20) becomes 2x(1) − 3x(2) − x(3) = 0.
Frequently, it is useful to think of the columns (or rows) of a matrix A as vectors.
These column (or row) vectors are linearly independent if and only if det A = 0.
Further, if C = AB, then it can be shown that det C = (det A)(det B). Therefore, if the columns (or rows) of both A and B are linearly independent, then the columns (or rows) of C are also linearly independent.
Now let us extend the concepts of linear dependence and independence to a set of vector functions x(1)(t), . . . , x(k)(t) defined on an interval α < t < β. The vectors x(1)(t), . . . , x(k)(t) are said to be linearly dependent on α < t < β if there exists a set of constants c , . . . , c , not all of which are zero, such that c x(1)(t) + · · · + c x(k)(t) = 0
1
k
1
k
for all t in the interval. Otherwise, x(1)(t), . . . , x(k)(t) are said to be linearly independent.
Note that if x(1)(t), . . . , x(k)(t) are linearly dependent on an interval, they are linearly dependent at each point in the interval. However, if x(1)(t), . . . , x(k)(t) are linearly independent on an interval, they may or may not be linearly independent at each point; they may, in fact, be linearly dependent at each point, but with different sets of constants at different points. See Problem 14 for an example.
Eigenvalues and Eigenvectors.
The equation Ax = y
(24)
can be viewed as a linear transformation that maps (or transforms) a given vector x
into a new vector y. Vectors that are transformed into multiples of themselves are important in many applications.4 To find such vectors we set y = λx, where λ is a scalar proportionality factor, and seek solutions of the equations Ax = λx, (25) or (A − λI)x = 0.
(26)
4For example, this problem is encountered in finding the principal axes of stress or strain in an elastic body, and in finding the modes of free vibration in a conservative system with a finite number of degrees of freedom.
7.3Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors
The latter equation has nonzero solutions if and only if λ is chosen so that (λ) = det(A − λI) = 0.
(27)
Values of λ that satisfy Eq. (27) are called eigenvalues of the matrix A, and the nonzero solutions of Eq. (25) or (26) that are obtained by using such a value of λ are called the eigenvectors corresponding to that eigenvalue.
If A is a 2 × 2 matrix, then Eq. (26) has the form
a − λ
a
x
0
11
12
1
=
(28)
a
a − λ
x
0
21
22
2
and Eq. (27) becomes (λ) = (a − λ)(a − λ) − a a = 0.
(29)
11
22
12 21
The following example illustrates how eigenvalues and eigenvectors are found.
Find the eigenvalues and eigenvectors of the matrix E X A M P L E
4
3
−1 A = .
(30)
4
−2
The eigenvalues λ and eigenvectors x satisfy the equation (A − λI)x = 0, or
3 − λ
−1
x
0
1
=
.
(31)
4
−2 − λ
x
0
2
The eigenvalues are the roots of the equation
3 − λ
−1
det(A − λI) =
4
−2 − λ = λ2 − λ − 2 = 0.
(32)
Thus the eigenvalues are λ = 2 and λ = −1.
1
2
To find the eigenvectors we return to Eq. (31) and replace λ by each of the eigenvalues in turn. For λ = 2 we have
1
−1
x
0
1
=
.
(33)
4
−4
x
0
2
Hence each row of this vector equation leads to the condition x − x = 0, so x and
1
2
1
x are equal, but their value is not determined. If x = c, then x = c also and the
2
1
2
eigenvector x(1) is
1
x(1) = c, c = 0.
(34)
1
Usually, we will drop the arbitrary constant c when finding eigenvectors; thus instead of Eq. (34) we write
1
x(1) = ,
(35)
1
and remember that any nonzero multiple of this vector is also an eigenvector. We say that x(1) is the eigenvector corresponding to the eigenvalue λ = 2.
1
Chapter 7. Systems of First Order Linear Equations
Now setting λ = −1 in Eq. (31), we obtain
4
−1
x
0
1
=
.
(36)
4
−1
x
0
2
Again we obtain a single condition on x and x , namely, 4x − x = 0. Thus the
1
2
1
2
eigenvector corresponding to the eigenvalue λ = −1 is
2
1
x(2) = ,
(37)
4
or any nonzero multiple of this vector.
As Example 4 illustrates, eigenvectors are determined only up to an arbitrary nonzero multiplicative constant; if this constant is specified in some way, then the eigenvectors are said to be In Example 4, we set the constant equal to 1, but any other nonzero value could also have been used. Sometimes it is convenient to normalize an eigenvector x by choosing the constant so that (x, x) = 1.
Equation (27) is a polynomial equation of degree n in λ, so there are n eigenvalues λ , . . . , λ , some of which may be repeated. If a given eigenvalue appears m times as
1
n
a root of Eq. (27), then that eigenvalue is said to have multiplicity m. Each eigenvalue has at least one associated eigenvector, and an eigenvalue of multiplicity m may have q
linearly independent eigenvectors, where 1 ≤ q ≤ m.
(38)
Examples show that q may be any integer in this interval. If all the eigenvalues of a matrix A are simple (have multiplicity one), then it is possible to show that the n
eigenvectors of A, one for each eigenvalue, are linearly independent. On the other hand, if A has one or more repeated eigenvalues, then there may be fewer than n linearly independent eigenvectors associated with A, since for a repeated eigenvalue we may have q < m. As we will see in Section 7.8, this fact may lead to complications later on in the solution of systems of differential equations.
Find the eigenvalues and eigenvectors of the matrix E X A M P L E
5
0
1
1
A = 1
0
1 .
(39)
1
1
0
The eigenvalues λ and eigenvectors x satisfy the equation (A − λI)x = 0, or
−λ
1
1
x
0
1
1
−λ
1 x = 0 .
(40)
2
1
1
−λ
x
0
3
The eigenvalues are the roots of the equation
−λ
1
1
det(A − λI) = 1 −λ 1 = −λ3 + 3λ + 2 = 0.
(41)
1
1
−λ
7.3Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors The roots of Eq. (41) are λ = 2, λ = −1, and λ = −1. Thus 2 is a simple eigenvalue,
1
2
3
and −1 is an eigenvalue of multiplicity 2.
To find the eigenvector x(1) corresponding to the eigenvalue λ we substitute λ = 2
1
in Eq. (40); this gives the system
−2
1
1
x
0
1
1
−2
1 x = 0 .
(42)
2
1
1
−2
x
0
3
We can reduce this to the equivalent system
2
−1 −1
x
0
1
0
1
−1 x = 0
(43)
2
0
0
0
x
0
3
by elementary row operations. Solving this system we obtain the eigenvector
1
x(1) = 1 .
(44)
1
For λ = −1, Eqs. (40) reduce immediately to the single equation x + x + x = 0.
(45)
1
2
3
Thus values for two of the quantities x , x , x can be chosen arbitrarily and the third
1
2
3
is determined from Eq. (45). For example, if x = 1 and x = 0, then x = −1, and
1
2
3
1
x(2) = 0 (46)
−1
is an eigenvector. Any nonzero multiple of x(2) is also an eigenvector, but a second independent eigenvector can be found by making another choice of x and x ; for
1
2
instance, x = 0 and x = 1. Again x = −1 and
1
2
3
0
x(3) = 1 (47)
−1
is an eigenvector linearly independent of x(2). Therefore in this example two linearly independent eigenvectors are associated with the double eigenvalue.
An important special class of matrices, Hermitian matrices, are those for which A∗ = A; that is, a = a . Hermitian matrices include as a subclass real j i
i j symmetric matrices, that is, matrices that have real elements and for which AT = A.
The eigenvalues and eigenvectors of Hermitian matrices always have the following useful properties:
1.
All eigenvalues are real.
2.
There always exists a full set of n linearly independent eigenvectors, regardless of the multiplicities of the eigenvalues.
Chapter 7. Systems of First Order Linear Equations3.
If x(1) and x(2) are eigenvectors that correspond to different eigenvalues, then (x(1), x(2)) = 0. Thus, if all eigenvalues are simple, then the associated eigenvectors form an orthogonal set of vectors.
4.
Corresponding to an eigenvalue of multiplicity m, it is possible to choose m
eigenvectors that are mutually orthogonal. Thus the full set of n eigenvectors can always be chosen to be orthogonal as well as linearly independent.
Example 5 above involves a real symmetric matrix and illustrates and 3, but the choice we have made for x(2) and x(3) does not illustrate property 4.
However, it is always possible to choose an x(2) and x(3) so that (x(2), x(3)) = 0. For example, in Example 5 we could have chosen
1
1
x(2) = 0 ,x(3) = −2
−1
1
as the eigenvectors associated with the eigenvalue λ = −1. These eigenvectors are orthogonal to each other as well as to the eigenvector x(1) corresponding to the eigenvalue λ = 2. The proofs of s3 above are outlined in Problems 32 and 33.
PROBLEMS
In each of Problems 1 through 5 either solve the given set of equations, or else show that there is no solution.
1.
x
2.
1
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
3.
2
4.
1
2
3
1
2
3
1
1
2
3
1
2
3
1
2
3
1
2
3
5.
x
1
3
1
2
3
1
2
3
7.3Systems of Linear Algebraic Equations; Linear Independence, Eigenvalues, Eigenvectors
14. Let
et
1
,
.
tet
t
In each of Problems 15 through 24 find all eigenvalues and eigenvectors of the given matrix.
5
−1
3
−2
15.
16.
3
1
4
−1
−2
1
1
i
17.
18.
1
−2
1
√
1
3
19.
√
20.
3
−1
−5
1
1
0
0
3
2
2
21. 2
1
−2
22. 1
4
1
3
2
1
−2 −4 −1
3
2
4
24. 2
0
2
4
2
3
Problems 25 through 29 deal with the problem of solving Ax = b when det A = 0.
25. Suppose that, for a given matrix A, there is a nonzero vector x such that Ax = 0. Show that there is also a nonzero vector y such that A∗y = 0.
26. Show that (Ax, y) = (x, A∗y) for any vectors x and y.
Hint: Use the result of Problem 26.
28. Suppose that det A = 0, and that x = x(0) is a solution of Ax = b. Show that if is a solution of A = 0 and α is any constant, then x = x(0) + α is also a solution of Ax = b.
29. Suppose that det A = 0 and that y is a solution of A∗y = 0. Show that if (b, y) = 0 for every such y, then Ax = b has solutions. Note that this is the converse of Problem 27; the form of the solution is given by Problem 28.
30. Prove that λ = 0 is an eigenvalue of A if and only if A is singular.
31. Prove that if A is Hermitian, then (Ax, y) = (x, Ay), where x and y are any vectors.
32. In this problem we show that the eigenvalues of a Hermitian matrix A are real. Let x be an eigenvector corresponding to the eigenvalue λ.
(a) Show that (Ax, x) = (x, Ax). Hint: See Problem 31.
(b) Show that λ(x, x) = λ(x, x). Hint: Recall that Ax = λx.
(c) Show that λ = λ; that is, the eigenvalue λ is real.
33. Show that if λ and λ are eigenvalues of a Hermitian matrix A, and if λ = λ , then the
1
2
1
2
corresponding eigenvectors x(1) and x(2) are orthogonal.
Hint: Use the results of Problems 31 and 32 to show that (λ − λ )(x(1), x(2)) = 0.
1
2