Matrices+ 02
Matrix Eigenvalues
The concepts addressed in this lecture for twodimensional real case are as well applicable to Ndimensional spaces and even to real or complex abstract vector spaces with linear transformations defined there.
Presentation in a twodimensional real space is chosen for its relative simplicity and easy exemplification.
Let's consider a 2⨯2 matrix A as a linear operator in the twodimensional Euclidean vector space. In other words, multiplication of any vector v on a coordinate plane by this 2⨯2 matrix A linearly transforms it into another vector on the plane w=A·v.
Assume, matrix A is
5  6 
6  10 
Let's see how this linear operator works, if applied to different vectors.
We will use a rowvector notation in the text for compactness, but columnvector notation in the transformation examples below.
The coordinates of our vectors we will enclose into double bars, like matrices, because a rowvector is a matrix with only one row, and a columnvector is a matrix with only one column.
Our first example of a vector to apply this linear transformation is v=1,1.





Applied to a different vector





Finally, let's applied our operator to a vector





As we see, for this particular matrix we found two vectors that, if transformed by this matrix as by a linear operator, retain their direction, while change the magnitude by some factor.
These vectors are called eigenvectors. For each eigenvector there is a factor that characterizes the change in its magnitude if this matrix acts on it as an operator. This factor is called eigenvalue. This eigenvalue in the example above was 1 for
There are some questions one might ask.
1. Are there always some particular vectors that retain the direction if transformed by some particular matrix?
2. If yes, how to find them and how to find the corresponding multiplication factors?
3. How many such vectors exist, if any?
4. How to find all the multiplication factors for a particular matrix transformation?
Let's analyze the linear transformation by a matrix that leaves the direction of a vector without change, just changes the magnitude by some factor λ.
Assume, we have a matrix A=a_{i,j}, where i,j∈{1,2}, in our twodimensional Euclidean space.
This matrix converts any vector
If matrix A transforms vector v to a collinear one with the magnitude of the original one multiplied by a factor λ, the following matrix equation must hold
A·v = λ·v
or in coordinate form







(a_{1,1}−λ)·v_{1} + a_{1,2}·v_{2} = 0
a_{2,1}·v_{1} + (a_{2,2}−λ)·v_{2} = 0
This is a system of two linear equations with three unknowns λ, v_{1} and v_{2}.
One trivial solution would be v_{1}=0 and v_{2}=0, in which case λ can take any value.
This is not a case worthy of analyzing.
If the matrix of coefficients of this system has a nonzero determinant, this trivial solution would be the only one.
Therefore, if we are looking for a nontrivial solution, the matrix's determinant must be zero, which gives a specific condition on the value of λ.
Therefore, a necessary condition for existence of other than nullvector v is
(a_{1,1}−λ)·(a_{2,2}−λ) − a_{1,2}· a_{2,1} = 0
or
λ² − (a_{1,1}+a_{2,2})·λ +
+ a_{1,1}·a_{2,2}−a_{1,2}·a_{2,1} = 0
Since we are looking for real values of λ, we have to examine a discriminant D of his quadratic equation.
D =
=(a_{1,1}+a_{2,2})²−4·(a_{1,1}·a_{2,2}−a_{1,2}·a_{2,1})
=(a_{1,1}−a_{2,2})²+4·a_{1,2}·a_{2,1}
If D is negative, there are no real solutions for λ.
If D is zero, there is one real solutions for λ.
If D is positive, there are two real solutions for λ.
Consider now that we have determined λ and would like to find vectors transformed into collinear ones by matrix A with this exact factor of change in magnitude.
If some vector v=v_{1},v_{2} that is transformed into a collinear one with a factor λ exists, vector s·v, where s is any real nonzero number, would have exactly the same quality because of associativity and commutativity of multiplication by a scalar.
A·(s·v) = (A·s)·v = (s·A)·v =
= s·(A·v) = s·(λ·v)= λ·(s·v)
Therefore, we don't need to determine exact values v_{1} and v_{2}, we just need to determine only the direction of vector
If v_{2}≠0, the directions of a vector v and that of vector v_{1}/v_{2},1 are the same.
If v_{1}≠0, the directions of a vector v and that of vector 1,v_{2}/v_{1} are the same.
From this follows that, firstly, we can search for eigenvectors among those with v_{2}≠0, restricting our search to vectors x=v_{1}/v_{2},1.
Then we can search for eigenvectors among those with v_{1}≠0, restricting our search to vectors 1,x=v_{1}/v_{2}.
In both cases we will have to solve a system of two linear equations with two unknowns λ and x.
Searching for vectors x,1
In this case the matrix equation that might deliver the required vector looks like this







a_{1,1}·x+a_{1,2}·1 = λ·x
a_{2,1}·x+a_{2,2}·1 = λ·1
Take the right side of the second equation λ and substitute into the right side of the first equation, obtaining a quadratic equation for x:
a_{1,1}·x+a_{1,2} = (a_{2,1}·x+a_{2,2})·x
or
a_{2,1}·x² + (a_{2,2}−a_{1,1})·x − a_{1,2} = 0
Two solutions for this equations x_{1,2}, assuming they are real values, produce two vectors x_{1},1 and x_{2},1, each of which satisfy the condition of collinearity after the matrix transformation.
Generally speaking, the factor λ will be different for each such vector.
Searching for vectors 1,x
In this case the matrix equation that might deliver the required vector looks like this







a_{1,1}·1+a_{1,2}·x = λ·1
a_{2,1}·1+a_{2,2}·x = λ·x
Take the right side of the first equation λ and substitute into the right side of the second equation, obtaining a quadratic equation for x:
a_{2,1}·1+a_{2,2}·x = (a_{1,1}·1+a_{1,2}·x)·x
or
a_{1,2}·x² + (a_{1,1}−a_{2,2})·x − a_{2,1} = 0
Two solutions for this equations x_{1,2}, assuming they are real values, produce two vectors 1,x_{1} and 1,x_{2}, each of which satisfy the condition of collinearity after the matrix transformation.
Generally speaking, the factor λ will be different for each such vector.
Once again, let's emphasize important definitions.
Vectors transformed into collinear ones by a matrix of transformation are called eigenvectors or characteristic vectors for this matrix.
The factor λ corresponding to some eigenvector is called eigenvalue or characteristic value of the matrix and this eigenvector.
Let's determine eigenvectors and eigenvalues for a matrix A
5  6 
6  10 
The quadratic equation to determine the multiplier λ for this matrix is
λ² − (a_{1,1}+a_{2,2})·λ +
+ a_{1,1}·a_{2,2}−a_{1,2}·a_{2,1} = 0
which amounts to
λ² − 15λ + 14 = 0
with solutions
λ_{1} = 1 and λ_{2} = 14
Let's find the eigenvectors of this matrix.
The quadratic equation for eigenvectors of type x,1 is
6x² + (10−5)x − 6 = 0 or
6x² + 5x − 6 = 0 or
Solutions are
x_{1,2} = (1/12)·(−5±√25+4·36) =
= (1/12)·(−5±13)
Therefore,
x_{1} = 2/3
x_{2} = −3/2
Two eigenvectors are:
v_{1} = 2/3,1 which is collinear to vector 2,3 used in the example above and
v_{2} = −3/2,1 which is collinear to vector 3,−2 used in the example above.
The matrix transformation of these eigenvectors are










Not surprisingly, both eigenvectors found above have eigenvalues already found (1 and 14).
The quadratic equation for eigenvectors of type 1,x is
6x² + (5−10)x − 6 = 0 or
6x² − 5x − 6 = 0 or
Solutions are
x_{1,2} = (1/12)·(5±√25+4·36) =
= (1/12)·(5±13)
Therefore,
x_{1} = 3/2
x_{2} = −2/3
Two eigenvectors are:
v_{1} = 1,3/2 which is collinear to vector 2,3 used in the example above and
v_{2} = 1,−2/3 which is collinear to vector 3,−2 used in the example above.
So, we did not gain any new eigenvalues by searching for vectors of a form 1,x.
The above calculations showed that for a given matrix we have two eigenvectors, each with its own eigenvalue.
Based on these calculations, we can now answer the questions presented before.
Q1. Are there always some particular vectors that retain the direction if transformed by some particular matrix?
A1. Not always, but only if the quadratic equations for x
a_{2,1}·x² + (a_{2,2}−a_{1,1})·x − a_{1,2} = 0
and
a_{1,2}·x² + (a_{1,1}−a_{2,2})·x − a_{2,1} = 0
where a_{i,j} (i,j∈{1,2}) is a matrix of transformation, have real solutions.
Q2. If yes, how to find them?
A2. Solve the quadratic equations above and, for each real solutions x of the first equation, vector x,1 is an eigenvector and, for each real solutions x of the second equation, vector 1,x is an eigenvector. Then apply the matrix of transformation to each eigenvector x,1 or 1,x and compare the result with this vector. It should be equal to some eigenvalue λ multiplied by this eigenvector.
Q3. How many such vectors exist, if any?
A3. As many as real solutions have quadratic equations above, but no more than two.
Incidentally, in threedimensional case our equations will be polynomial of the 3rd degree, and the number of solutions will be restricted to three.
In Ndimensional case this maximum number will be N.
Q4. How to find all the multiplication factors for a particular matrix transformation?
A4. Quadratic equation for eigenvalues
λ² − (a_{1,1}+a_{2,2})·λ +
+ a_{1,1}·a_{2,2}−a_{1,2}·a_{2,1} = 0
can have 0, 1 or 2 real solutions.
The concept of eigenvectors and eigenvalues (characteristic vectors and characteristic values) can be extended to Ndimensional Euclidean vector spaces and even to abstract vector spaces, like, for example, a set of all real functions integrable on a segment 0,1.
The detail analysis of these cases is, however, beyond the current course, which aimed, primarily, to introduce advance concepts.
Problem A
Research conditions when a diagonal matrix (only elements along the main diagonal are not zero) has eigenvalues.
Solution A
Matrix of transformation A=a_{i,j} has zeros for i≠j.
So, it looks like this
a_{1,1}  0 
0  a_{2,2} 
The equation for eigenvalues in this (a_{1,2}=a_{2,1}=0) case is
λ² − (a_{1,1}+a_{2,2})·λ + a_{1,1}·a_{2,2} = 0
with immediately obvious solutions
λ_{1}=a_{1,1} and λ_{2}=a_{2,2}
So, the values along the main diagonal of a diagonal matrix are the eigenvalues of this matrix.
Determine the eigenvectors now among vectors x,1.
Original quadratic equation for this case is
a_{2,1}·x² + (a_{2,2}−a_{1,1})·x − a_{1,2} = 0
With a_{2,1}=a_{1,2}=0 it looks simpler:
(a_{2,2}−a_{1,1})·x = 0
From this we conclude that, if a_{2,2}≠a_{1,1}, the only solution is x=0, so our eigenvector is 0,1.
The eigenvalue for this eigenvector is a_{2,2}.
If a_{2,2}=a_{1,1}, any x is good enough, so any vector is an eigenvector.
Determine the eigenvectors now among vectors 1,x.
Original quadratic equation for this case is
a_{1,2}·x² + (a_{1,1}−a_{2,2})·x − a_{2,1} = 0
With a_{2,1}=a_{1,2}=0 it looks simpler:
(a_{1,1}−a_{2,2})·x = 0
From this we conclude that, if a_{2,2}≠a_{1,1}, the only solution is x=0, so our eigenvector is 1,0.
The eigenvalue for this eigenvector is a_{1,1}.
If a_{2,2}=a_{1,1}, any x is good enough, so any vector is an eigenvector.
Answer A
If matrix of transformation is diagonal
a_{1,1}  0 
0  a_{2,2} 
the two eigenvectors are base unit vectors and the eigenvalues are a_{1,1} for base unit vector 1,0 and a_{2,2} for base unit vector 0,1.
In the case of a_{1,1}=a_{2,2} any vector is an eigenvector with eigenvalue a_{1,1}.
Problem B
Prove that symmetrical matrix always has real eigenvectors.
Solution B
Matrix of transformation A=a_{i,j} is symmetrical, which means a_{1,2}=a_{2,1}.
Recall that a necessary condition for existence of real eigenvalues λ is
(a_{1,1}−λ)·(a_{2,2}−λ) − a_{1,2}· a_{2,1} = 0
or
λ² − (a_{1,1}+a_{2,2})·λ +
+ a_{1,1}·a_{2,2}−a_{1,2}·a_{2,1} = 0
Since we are looking for real values of λ, we have to examine a discriminant D of his quadratic equation.
D = (a_{1,1}−a_{2,2})²+4·a_{1,2}·a_{2,1}
Since a_{1,2}=a_{2,1}, their product is nonnegative, which makes the whole discriminant nonnegative.
If D is zero, there is one real solutions for λ.
If D is positive, there are two real solutions for λ.
So, one or two solutions always exist.