aATA and AAT

1. Property-1 2. Property-2 3. Property-3 4. Property-4 5. Property-5 6. Property-6 7. Property-7 8. Property-8

If A is a m×n matrix, then ATA is a n×n matrix and AAT is a m×m matrix. 1. Property-1

ATA and AAT are symmetric matrices.

Let us show this for ATA: (ATA)T=AT(AT)T=ATA The argument is similar for AAT. 2. Property-2

The nullspace of A is equal to the nullspace of ATA. Likewise, the nullspace of AT is equal to the nullspace of AAT.

Let us denote the nullspaces as N(A) and N(ATA). • If x∈N(A), then we have the following sequence of observations: a

Ax	=0
ATAx	=0
(ATA)x	=0

Thus, x∈N(ATA) and therefore N(A)⊆N(ATA). • If x∈N(ATA), then we have the following sequence of observations: a

(ATA)x	=0
xTATAx	=0
(Ax)T(Ax)	=0
Ax	=0

Thus, x∈N(A) and therefore N(ATA)⊆N(A). From this, we conclude that N(A)=N(ATA). A similar argument can be used for N(AT)=N(AAT). 3. Property-3

rank(ATA)=rank(AAT)=rank(A)

Using the rank nullity theorem, we see the following: a

rank(A)+nullity(A)	=n
rank(ATA)+nullity(ATA)	=n

Using ?, we can conclude that the rank of ATA is the same as the rank of A. Next, we know that the row rank is the same as the column rank, or rank(A)=rank(AT). Again applying the rank nullity theorem: a

rank(AT)+nullity(AT)	=m
rank(AAT)+nullity(AAT)	=m

We can now conclude that rank(ATA)=rank(AAT)=rank(A). 4. Property-4

If rank(A)=n, then ATA is invertible.

Note that if rank(A)=n, then the nullity of A is zero. Since A and ATA have the same nullspace, nullity of ATA is also zero. From this it follows that the rank of ATA is n. Since ATA is a full rank matrix, it is invertible. 5. Property-5

All the eigenvalues of ATA and AAT are non-negative.

Let (𝜆,u) be an eigenpair of AAT. Then: (1)a

(AAT)u	=𝜆u
uTAATu	=𝜆uTu
(ATu)T(ATu)	=𝜆||u||2
||ATu||2	=𝜆||u||2

Since u is an eigenvector of AAT, it is not the zero vector, so ||u||≠0. We can now express 𝜆 as: 𝜆=||ATu||2

||u||2⩾0 A similar argument holds for ATA. 6. Property-6

ATA and AAT have the same non-zero eigenvalues.

Let (𝜆,v) be an eigenpair of ATA with 𝜆≠0. a

(ATA)v	=𝜆v
A(ATA)v	=𝜆(Av)
(AAT)(Av)	=𝜆(Av)

(𝜆,Av) is an eigenpair of AAT provided Av≠0 as an eigenvector cannot be the zero-vector. We need to show that Av≠0. If Av happens to be 0, then (ATA)v will also become 0. But this would mean that v is an eigenvector of ATA with eigenvalue 0, implying that 𝜆=0. This contradicts the fact that 𝜆≠0. So, Av≠0 and hence is an eigenvector of AAT. We have shown that every non-zero eigenvalue of ATA is also an eigenvalue of AAT. We can now go the other way and show that every non-zero eigenvalue of AAT is also an eigenvalue of ATA. 7. Property-7

This is a result that lets us move between the eigenvectors of ATA and AAT for the positive eigenvalues. There are two similar looking results: (1) If (𝜆,v) is an eigenpair for ATA with 𝜆>0 and ||v||=1, then (𝜆,u) is an eigenpair for AAT where• u=Av

𝜎 with 𝜎=𝜆• ||u||=1 (2) If (𝜆,u) is an eigenpair for AAT with 𝜆>0 and ||u||=1, then (𝜆,v) is an eigenpair for ATA where• v=ATu

𝜎 with 𝜎=𝜆 • ||v||=1

Let us prove this for (2): a

(AAT)u	=𝜆u
AT(AAT)u	=𝜆ATu
(ATA)(ATu)	=𝜆(ATu)

From ? we already know that (𝜆,ATu) is an eigenpair of ATA. We can normalize ATu by dividing it by ||ATu||, which we shall compute now: a

||ATu||2	=(ATu)T(ATu)
	=uTAATu
	=uT[(AAT)u]
	=uT(𝜆u)
	=𝜆
⟹||ATu||	=𝜆
	=𝜎

8. Property-8

AAT and ATA have exactly r positive eigenvalues, where r is the rank of A.

This proof requires an understanding of the concept of multiplicity of eigenvalues. Let us take the case of ATA. Its rank is r and its nullity is n-r. The nullity is the dimension of the eigenspace corresponding to the eigenvalue 0. In other terms, the geometric multiplicity of the eigenvalue 0 is n-r. Since ATA is symmetric, it is diagonalizable, a result we know from the spectral theorem. For a diagonalizable matrix, the geometric multiplicity is equal to the algebraic multiplicity for every eigenvalue. So the eigenvalue 0 will occur exactly n-r times. This means that the remaining r eigenvalues, not necessarily distinct, should all be positive.