Tip:
Highlight text to annotate it
X
In the last lecture, we looked at the properties of a positive semi definite matrix. Now use
all this ideas as we mentioned at the end of the last lecture; we should now use all
the ideas that we have developed so far in this course to analyze a general given matrix.
So recall the properties of a positive semi definite matrix; one - all Eigen values are
real and this comes from the fact that a positive semi definite matrix is always Hermitian;
the second point is that all Eigen values are greater than that or equal to zero. And
this is that if nu A is the nullity of A and rho is the rank of A, where A is positive
definite; A is positive semi definite. Then its Eigen values can be arranged as in a decreasing
order Eigen value than the next smaller Eigen value and so on. These are all greater than
zero; rho of them are positive Eigen values and the remaining or all there are nu A of
them; these are all zero Eigen values.
And then corresponding to this we have orthonormal Eigen vectors. We have V 1, V 2, V rho corresponding
to the Eigen values lambda 1 lambda 2, lambda rho the positive Eigen values. So, we have
the rho orthonormal Eigen vectors corresponding to the positive Eigen values and then phi
1, phi 2, phi nu A corresponding to the Eigen value zero. So, the Eigen vectors can be found
and we also found that V 1, V 2, V rho provide an orthonormal basis for the range of A. So,
these are all the fundamental properties of a positive semi definite matrix.
Now, we shall look at a general matrix and see how we are going to use these properties
to analyze a general matrix. So, let us now start with the analysis of a general m by
n matrix. Now without loss of generality, we shall look at A at the real m by n matrix
and we will point out what are the minor changes that we have to make whenever we deal with
a complex m by n matrix.
So first, we look at the real m by n matrices. Recall that corresponding to A, we have four
fundamental sub spaces. Two of them namely range of A transpose and the null space of
A sub spaces of R n, and the range of A and the null space of A transpose sub spaces of
R m and these pairs are orthogonal complements of each other orthogonal complements of each
other. What this means is, we have the space R n; A maps vectors n component vectors to
m component vectors and A transpose take m component vectors n component vectors and
these space is… In this space, we have this two orthogonal complements. This is the range
of A transpose; this is the null space of A. On this side we have the two orthogonal
complements; the range of A and the null space of A.
The fundamental problem is choosing suitable basis for this. Notice that the dimension
of this is the dimension of the null space of dimension of the range of a transpose which
is the rank of A transpose which is the same as the rank; will denote the rank of the matrix
by rho. Similarly, the dimension here is the nullity; here the dimension is again the rank;
here the dimension in the dimension nullity of A transpose.
So, these are all the dimensions of these four spaces. The two spaces the range A transpose
in the range A at the same dimension namely the rank of the matrix; null space of a dimension
nu A; the null space of A transpose is the nullity of nu A transpose.
So the basic problem is to choose orthonormal bases for these four sub spaces in a suitable manner which makes the analysis
of the matrix A easy. We shall see the meanings of the word suitable easy etcetera as we go
along. This is the fundamental problem; the fundamental problem is to choose the sub spaces
these four fundamental sub spaces we have got them. So, we have split the two vectors
spaces R n and R m into two parts each. Now in each part we are going to do the sampling
namely get the basis. When we get the basis, we want at always be orthonormal. So, the
competitions become easy and we want to choose this basis in such a way that it makes our
competitions and analysis easy. So, it is for this purpose of choosing the suitable
basis of suitable of suitable basis choosing. We use the ideas of positive semi definite
matrices. How do we do this? That is the question. So, what we do is even though the matrix given
matrix may be rectangular or it may be square we do not know whether it is hermitian it
may or may not be hermitian; it may or may not be positive semi definite. What we will
do is starting with the given matrix A we shall construct another matrix which is positive
semi definite in such a way the analysis of the positive semi definite matrix that we
construct will reflect in the analysis of the given matrix A.
So given A in R m n, we construct a positive semi definite matrix which will call us L
which is n could as n such that the analysis of L reflects in the analysis of A. Now, the
analysis of L will be easy because it is positive semi definite and we have seen all the properties
of positive semi definite matrices. Now, how do we define this matrix L?
So, we define L to be A transpose A. Notice A is m by n and A transpose is n by m. So,
the product is n by m. So, L belongs to n by m. So, first property of L is that L belongs
to n cross m. So, L is an n cross n matrix. So, even though the matrix original matrix
A may not be rectangular we have construct may be rectangular and may not be square,
we still constructed a square matrix out of it which is L and which is an n by n square
matrix. So, the moment we have a square matrix we look at its properties now. The second
property that we look at is the following. Suppose, x belongs to R n then we have Lx
comma x is equal to A transpose Ax comma x which is equal to (Ax, Ax). Remember, when
you move A transpose to the second factor you will go with the another transpose. So,
it will become A transpose transpose, which is A which is equal to the length of Ax square
which is greater than or equal to zero.
So therefore, Ax comma sorry therefore, Lx,x is greater than or equal to zero for every
x in R n. This is preciously the meaning of the fact or this is preciously saying that
L is a positive semi definite matrix. So, L is a positive semi definite matrix belonging
to R n. So given an n by n m by n matrix, we can always construct a positive semi definite
matrix which is n by m. Analogously, we can define m to be AA transpose and this is a
positive semi definite matrix in R mm. So given the rectangular m by n matrix, we have
constructed two positive semi definite matrices. One of them is n by n; the other one is m
by n. We shall study the properties of L analogously will get the properties of m.
Now given L, what does we do? This is now, we have constructed this L; now take this
matrix L, what does this do? If an n by n matrix, so it maps, it takes R n vectors to
R n vectors. Now what is L transpose?
Since it is positive definite it we make sure that it is symmetric and by very definition,
we see that L transpose is equal to l and therefore, L is symmetric real symmetric.
So, L transpose is L; so it is symmetric matrix. So L transpose also maps R n to R n. So, the
reverse map R n to R n L transpose is the same as L so they we do not get anything new.
Now corresponding to L, we must have a decomposition of R n. Remember, that the moment we have
a matrix, we have the subspaces; what are the subspaces for L?
The subspaces of L or one range of L null space of L than we have to say range of L
transpose, but L transpose is L so that is the same as this so nothing new. Similarly,
L transpose is L; therefore, null space of L transpose is the same as null space of L.
So, there are two basic subspaces of R n; namely the range of L and the null space of
L. So, these are the two subspaces we get. Actually, I will write or I will transpose
which is equal to R n. We know that the null space of any matrix is orthogonal complement
of the range of the L transpose. But in this case, the transpose is the original matrix;
therefore, the null space of L is the orthogonal complement of the null space of m or similarly,
we get for the matrix m on this side the decomposition, because it is an n by n matrix; it decomposes
R m by R m both this space sides are the same. It is the range of m transpose which is same
as m everything and this is the null space of m transpose, which is the same as null
space of m.
So, we have one decomposition of R n given by L and one give decomposition of R m given
by m. So now, if you look at R n and R m, we have the decomposition on this side given
by the range of L and the null space of L and the decomposition into orthogonal complements
on this guide given by range of m and the null space of m.
On the other hand, A also gives a decomposition and the other hand, we have R n and R m we
have the decomposition given by A which is range of A transpose and null space of A on
this side and range of A range of A and null space of A transpose on this side. Now, let
us look at this two pairs of decomposition and focus on R n first. On the R n side, we
have one decomposition given by L; namely R L and N L. Also another decomposition given
by A; namely R A transpose and N A so on.
For R n we have two orthogonal complement decompositions. One of them is the range of
L and N L the null space of L given by L and the other is the range of A transpose and
the null space of A given by A. So, we have this two decomposition of the same space;
one arising out of the matrix L and the other arising out of the matrix A, but the matrix
L is connected with the matrix A, because it is defined from the matrix A starting from
the matrix A we defined L as a transpose A; so, we expect there must be some connection
between these two decomposition.
So is there any connection between these two decompositions? We shall now investigate this
question. So, to do thus to look at this we shall first look at some properties of the
null space of L and the range of L and the range of L. So, we shall now look at the null
space and the range of L. So first, the null space of L; suppose, we have a vector x which
is the null space of L. Remember, L is a n by n matrix so the null space of L is a part
of R n. So, x is in R n. So x belongs to R n and x belongs to null space of L implies
Lx since x is n by n x is n by 1; L is n by n; Lx also belong to R n and since it is the
null space, it must be the zero vector of that space. So, Lx is equal to theta n. If
Lx is equal to theta n that says Lx comma x the inner product of Lx with x is the same
us the inner product of theta n with x which is zero. So that says, A transpose Ax comma
x is zero because, L is define to be A transpose A.
Now in an inner product if you move the A’s from one side to the other side it gets added
up with the transpose. So, remove it to the second factor, it becomes A transpose transpose;
that is it becomes Ax, Ax is equal to zero. That says the length of Ax square is zero.
Now, note that A is an m by n matrix; x is an n by 1 vector. So Ax is a vector in R m.
Its length is zero and therefore, Ax must be the zero vector of the R m space; which
means, x is in the null space of A.
So, what we have seen is that whenever you have a vector in the null space of L, it must
also be in the null space of A. So therefore, null space of L is contained in null space
of A. So, we have started first look a at some connection between these two pieces then,
null spaces of L and A and the range of L and A A transpose. So, this is the first property;
let us call it as 1. On the other hand, x belongs to null space of A implies Ax since
A is m by n and x is n by 1 that is the zero vector in the m space. Now, if I multiply
both sides by A transpose, I get A transpose theta m which is A is transpose is n by m;
this is m by 1 so it will be the zero vector of the n space.
So that says, Lx is theta n because A transpose A is L. So that says, x is in the null space
of L. So consequently, we have the null space of A is part of null space of L because, anything
in the null space of A is also in the null space of A. Compare 1 and 2, we get we get
null space of L is the same as null space of A. So now, if you look at this picture,
the two decompositions on the R n side this and this are the same. If these two are the
same their corresponding orthogonal complements must be same. So basically, these two decompositions
collapse into the same decomposition. So, we shall now put this together.
So, we have the null space of L is equal to null space of A. Consequently, we have the
range of A range of L is the same as range of L transpose, but for any matrix the range
of the transpose is the orthogonal complement of the null space, but the null space of L
transpose is the same as null space of L perpendicular because, L is symmetric real symmetric L transpose
is equal to L. This is what we have used here this is what we use here because L transpose
is equal to L and because L transpose is equal to L. Now, N L perp is the same as N A perp,
because we are just seen that N L is the same as N A, but N A perp is the range of A transpose
perp.
So therefore, hence we get range of L is the same as range of A transpose perp. So hence,
the decomposition of R n into orthogonal complements given by given by L and A are the same. So,
we have this picture on the R n site. We have we call it null space of A is the same as
null space of L and we have the range of A transpose which is the same as the range of
n
Consequently, we get dimension of N L is the same as dimension of N A, but the dimension
of N L is the nullity of L and the dimension of N A is the nullity of A. So, nullity of
L is the same of nullity of A.
Similarly, because R A transpose is equal to R A R L, we have dimension of the range
of A transpose. If the dimension of the range of L the dimension of range A transpose is
the rank of A transpose and this is the rank of L, but rank of A, which we denote by rho
A transpose, but the rank of A transpose is the same as rank of A which we have seen.
So, the rank of A is equal to rank of L. So, the L at the same rank as A and same nullity
as A; so, what are the conclusions that we have? The nullity of A is the nullity of L;
the rank of L at the rank of A is equal to rank of L is also equal to rank of A transpose.
So, these are the properties that the fact that the decomposition given by the A and
the by given by L of R n is the same.
Analogously on the decomposition of R m we have on the R m site, we have the null space
the range of A which will be same as range of m now. Remember, m is AA transpose and
the null space of A transpose will be the same as null space of m.
So putting all these pictures together, we have R n R m the two spaces; A goes this side;
A transpose goes this side and on the R n site we have the decomposition range of A
transpose which is the same as range of L. Null space of A transpose which is same as
null space of null space of A which is the same as null space of L. On this side, we
have the range of A which is the same as range of m and null space of A transpose which is
the same as null space of m. Again you looking at the right hand side, we see that the dimension
of the null space of A transpose is the same as the dimension of the null space of m which
gives as the nullity of A transpose is equal to nullity of m.
Similarly, if we look at the decomposition of the R m, we get dimension of range of A
is the same as dimension of range of m. This is the rank of A, which is the same as rank
of m. So, we have seen that the rank of A the same of rank of L, which is the same as
rank of A transpose.
So we therefore have the following situations. The nullity of A is nullity of L; the nullity
of A transpose is equal to the nullity of m. The rank of A is the same as rank of A
transpose with same of rank of L the same of rank of m. All these matrices AA transpose
L and m share the same rank, whereas the nullities will depend on A and A transpose; L will have
the same nullity as A and m will have the same nullity of A transpose. Could these are
the four fundamental relations. The relation between the sub spaces is described in this
picture. The range of A transpose is same of range of L; null space of A is the same
as null space of L; the range of A is the same as range of m; the null space of A transpose
is the same as null space of m. This give rise to the following relationship
between the nullities and the rank of these various matrices; important think to notice
that AA transpose L m all share the same rank that number associate with the matrix, this
is the very important number.
Now, let us see how we are going to exploit to find the basis for the four sub spaces
now. So, how do we use these to find orthonormal basis for these four spaces that is the main
question. Now, let us look at first the null space of A same as null space of L, but L
is positive semi definite and therefore, null space of L basis orthonormal is given by the
orthonormal Eigen vectors; let us call them phi, 1 phi 2 the nullities nu A is the same
as nu L. So, instead of writing nu L, we will write nu A. Note b note nu A equal to nu L.
Orthonormal Eigen vectors corresponding to Eigen value zero. Recall that we have seen
that the null space basis for a positive semi definite matrix corresponds to the Eigen vector
sub zero. So, we can find the orthonormal basis for N A through the positive semi definite
matrix L and from its Eigen vectors corresponding to the Eigen value zero, we can find the orthonormal
basis for the null space of A. So, we have finish one-fourth of our job; namely finding
the Eigen was orthonormal basis for N A we are able to get.
Similarly, we look at N A transpose orthonormal basis corresponds to the orthonormal Eigen
vectors corresponding to the Eigen value zero of the matrix m; here we should write here
it was L of the matrix L. So, we have now for we have now seen out of this four sub
spaces the this the null space part here can be founded from the zero Eigen vector of L,
and the null space part on this side can be found out from the zero vector correspond
zero Eigen value corresponding to the matrix m. So, let us denote this basis by psi 1,
psi 2 and it will have nu A transpose vectors because, nu A transpose is equal to nu m.
So now, you have this picture. Again let us keep on writing this picture until we complete
this whole thing with the basis; we have R n; we have R m; we have A going this way;
A transpose going this way and this is the null space of A which is the same as null
space of L. Now, we have found a basis for this and then, this is the null space of A
transpose which is the same as null space of m, we have found a basis for this. So now,
our job is to find the basis for these two fellows. This is the range of A transpose
which is the range of L; this is range of A which is the range of m. So, these are two
things that we have to find.
Now, let us look at the range of A transpose which is the same as the range of L. Now,
L is positive semi definite; its positive Eigen values can be arranged as lambda 1 greater
than or equal to lambda 2 and how many of them will be there that will exactly be the
rank of the matrix A. We will not write rho sub A because, rho sub A is the same as rho
sub A transpose the same as rho L is the same as rho m. This all of them share the same
rank; we will not distinguish and simply write; all these are the Eigen values greater than
zero and the zero Eigen value has been taken care of here then finding the null space and
we get we know that corresponding to the positive Eigen values, we get orthonormal Eigen vectors
when we analyzed positive semi definite matrices, we found that corresponding to the positive
Eigen values we get orthonormal Eigen vectors V 1 corresponding to lambda 1; V 2 corresponding
to lambda 2; V rho corresponding to lambda rho.
So we found that at the end of the last lecture, we found that the V 1, V 2, V rho which are
the orthonormal Eigen vectors corresponding to the positive Eigen values of a positive
semi definite matrix give rise to a basis for the range of that positive semi definite
matrix. So, we found that V 1, V 2, V rho provide an orthonormal basis for range of
L because, L is positive semi definite, but the range of L is the same as range of A transpose.
So now, we have the picture which is progressing L in the sense that we had R n; we have R
m and A goes this way and A transpose goes this way and we have the range of A transpose
which is the same as range of L, and we have the null space of A which is the same as null
space of L, and for this we found the basis. This comes from the Eigen value zero of L
and now we found the orthonormal basis V 1, V 2, V rho which come from the positive Eigen
values L and the corresponding orthonormal Eigen vectors and on this side, we had the
null space of A transpose which is the same as null space of m we found orthonormal basis
which corresponds to the zero Eigen value of m and the Eigen vectors orthonormal Eigen
vectors corresponding to the zero Eigen values of m. Now, the only thing that remains is
finding an orthonormal basis for the range of A or the range of m. Now, we will show
a clever method of choosing this in order that this is where the cleverness of choosing
the basis come will choose the basis for the range of A in such a way that from then on
our analysis becomes easy. Now, how do you choose this basis?
So, we have now the last part is to choose an orthonormal basis for range of A which
is the same as range of m. Once we choose that, we have the four subspaces for orthogonal
subspaces two on the one side, and two on the other side. We have the orthonormal basis
from them and then we shall analyze everything in terms of this orthonormal basis. Now, we
have V 1, V 2, V rho orthonormal basis for range of A transpose which is the same as
range of L, and these are orthonormal Eigen vectors corresponding to the positive Eigen
values lambda 1 greater than lambda 2 greater than or equal to lambda rho of L and therefore,
hence L of V j is equal to lambda j V j for j equal to 1, 2 up to rho. Let us call this
equation as 1. So, we have this rho Eigen vectors of L, which form an orthonormal basis
for the range of A transpose.
Now, we define we look at what happens to V j under A so look at AV j. Now, V j is in
R n so it is n by 1; A is m by n so that belongs to R m. So certainly, this a vector in R m.
Secondly, it is of the form A of something so it belongs to the range of A. So, AV j
are all in range of A; j equal to 1, 2, rho. Any vector of the form A of some vector must
be in the range of A so these vectors are AV j. So, if we define W j equal to AV j;
j equal to 1, 2, rho then W j all belong to range of A. So, we have captured some vectors
in the range of A rho of them are there if you are lucky they will themselves form a
basis for the range of A and if you are very lucky, they may even form an orthonormal basis
for the range of A. Now, let us see whether they do this.
Now, will W j form a basis for range of A? Will W j form an orthonormal basis for range
of A? If so, our search for the basis for the range of A is over; so let us look at
w j. So, we look at these W j. We have W j comma W r the inner product of W j with W
r; W j by definition was AV j; W j is AV j. Similarly, W r is AV r. So therefore, W j,
W r inner product is the same as AV j AV r inner product. Once again, we observe that
in the real matrix case when we move the matrix from the inner product from one factor to
another factor it goes with the transpose. So, it becomes A transpose AV r. Now, A transpose
A by definition is L so that is LV r. Now recall that this V 1, V 2, V r are Eigen vectors
corresponding to the positive Eigen values and therefore, LV r will be equal to lambda
r V r.
So, this will be equal to V j lambda r V r. Now, when we pull out a constant lambda r
from the second factor of the inner product everything is real. The Eigen values are all
real because positive semi definite so we can pull it out as V j comma V r. So, W j
comma W r is the same as lambda r times V j comma V r, but this V 1, V 2, V j are orthonormal
and therefore, V j, V r will be equal to zero if j is not equal to r and 1 if j equal to
r. So therefore, this will be zero if j is not equal to r and when j equal to r, V j
V r is 1 because, every vector has length one and multiplied by lambda. So, it is equal
to lambda r if j equal to r; lambda r or lambda j which ever we want to write pull it.
So therefore, this fact says W j, W 1, W 2, W rho are orthogonal to each other if we take
any two of them any two of them there are orthogonal, but they do not have length 1.
So, what this says is, that W 1, W 2, W rho are orthogonal to each other and the length
of each one of them is equal to squared is lambda j because W j W j is lambda j in our
calculate. Put r equal to j you get W j W j and that is equal to lambda j. So, W length
of W j squared is lambda j. If we have a set of orthogonal vectors and if you divide each
one of them by its corresponding length you automatically get an orthonormal set of vectors.
So now, we define u j to be equal to W j by its length; its length is W j by lambda j
square root because, the length of j square W j squared is lambda j. So, the length of
W j is square root of lambda j. Now recall that this lambdas are the positive Eigen value
that we have talking about and therefore, there is no problem about the taking the square
root of positive quantities. Now, the question is which square root we take, as a convention
we take the non negative or the positive square root.
So, where we take positive square root of lambda, we have to take the positive square
root because we want it to be length of W j. So now, we denote by s j square root of
lambda j and these are called the singular values of A. The s 1, s 2, s rho are called
the singular values of A.
So now, what do we have we have u j therefore, is W j by s j then u 1, u 2, u rho are orthonormal
vectors m range of A, but range of A has dimension rho. So, if we have a rho dimensional space
and we have rho orthonormal vectors; that means, they form an orthonormal basis form
an orthonormal basis for range of A. So, we also got the orthonormal basis for range of
A. Note we have u j is equal to W j by s j; by W j by definition was AV j and therefore,
we get AV j is equal to s j u j; part j equal to 1, 2, rho.
On the other hand, we have u j belongs to R m; therefore, we can take A transpose u
j; A transpose u j is the same as a transpose AV j by s j because, u j is AV j by s j. Now
A transpose A is LV j; A transpose A is l. So, A transpose A is V j is LV j, but V j
is an Eigen vector corresponding to the Eigen value. So, it is lambda V j by s j, but s
j is square root of lambda j. So, it is just square root of lambda j V j because s j is
equal to square root of lambda j, but that is the same as s j V j. So therefore, we have
a transpose u j is s j V j. So, what is the structure now?
So, we have this R n; we have the m dimensional space; A is a mapping from R n to R m; A transpose
takes R m vectors to R m. We have the subspaces here range of A transpose which is the same
as range of L; null space of A which is the same as null space of L. We have the orthonormal
basis V 1, V 2, V a rho; we have the orthonormal basis phi 1, phi 2, phi nu A and then, we
have on this side the range of A which is same as range of m; null space of A transpose
which is the same as null space of m. We have the orthonormal basis u 1, u 2, u rho and
psi 1, psi 2, psi nu A transpose.
And the fundamental relation is that AV j is s j u j and A transpose u j is s j V j;
what this means is that the basis vector V 1 here; the basis vector V 1 on this side
goes to the same direction as the basis vector u 1 with the scaling factor s 1 because, AV
j has the scaling factor s j. So, V 1 goes to the u 1 direction with the scaling factor
s j; V 2 goes to the u 2 direction with the scaling factor s 2; and V rho goes to the
u rho direction with scaling factor s rho. So, basis directions are match to basis direction
expect that there is a certain dilation taking place in each one of these basis direction.
So, V j is put to the u j direction with the scaling s j; V 1 is put to the u 1 direction
with the scaling s 1; V 2 is put to the u 2 direction with the scaling s 2 and so on.
So, this is for j equal to 1, 2 up to rho; these connection between these. So, now we
have constructed a basis for the range of A transpose and the range of A in such a way
that the basis direction are connected that is the first basis vector goes to the same
at the direction of the first basis vector on R m side; second basis vector of range
of A transpose go to the same direction as the second basis of the range of A and so
on and so forth. So, the direction the basis let us map to basis expect there is a scaling
factor. So, in our to get the orthonormalization we have to do this scaling factor. So, thus
we have using the positive definiteness of the matrices L and m we have constructed,
we have been able to get the orthonormal basis for these four subspaces.
Now, all of them come out have the Eigen vectors as the orthonormal Eigen vectors corresponding
to the positive semi definite matrix L on this side and the positive semi definite matrix
on the m on that side. So, all the competitions therefore are reduced to the competition of
a positive semi definite matrix and therefore, whether the given matrix is rectangular or
square; whether it is even if it is square whether it is hermitian or not, we can always
construct a positive definite matrices semi definite matrices L and m starting from the
given matrix A from which we can construct the orthonormal basis for all the four subspaces
that we want and the orthonormal basis for the range of A transpose in the range of A
or constructed in such a way they are link to each other.
The moment we know the orthonormal basis V 1, V 2, V rho for the range of A transpose,
we can extract the orthonormal basis u 1, u 2, u rho from this relation AV j equal to
s j nu j. So, thus we have chosen certain basis in a in a connected manner and we shall
now see how we are going to and use all this basis to analyze a given matrix. As a feel
for it the idea is we are trying to solve a system of equation Ax equal to b; the vector
b lies on the R m side; since it lies on the R m side that is the known vector. It lies
on the r m side; it can be expanded in terms of the orthonormal basis u 1, u 2, u rho psi
1, psi 2, nu A transpose. The unknown vector x we have trying to find out lies on this
side and therefore, it can be expanded in terms of this orthonormal basis V 1, V 2,
V rho so and phi 1 phi 2.
So, we make that the observation, we have V 1, V 2, V rho phi 1, phi 2, phi nu A is
an orthonormal basis for the space R n and u 1, u 2, u rho psi 1, psi 2, psi nu A transpose
is an orthonormal basis for R m and the fundamental relation is A u j is equal to s j V j and
A tran AV j is equal to s j u j and A transpose u j is equal to s j v j for 1 less than or
equal to j less than or equal to rho; s j is equal to square root of lambda j; the lambda
1, lambda 2, lambda rho are the positive Eigen values of L. Now, we shall see how we use
this four basis to analyze a given general matrix? And this analysis, we will begin in
the next lecture.