Matrix Algebra
Matrix operations · The inverse of a matrix · Characterizations of invertible matrices
By the end of this week you will be able to:
- add, scale, multiply and transpose matrices, and decide when a product is defined;
- use the algebra rules for matrices and point out where they differ from the rules for numbers;
- decide if a \(2\times 2\) matrix is invertible with its determinant, and write its inverse;
- find the inverse of an \(n\times n\) matrix by row reducing \([\,A \mid I\,]\), and use an inverse to solve \(A\mathbf x=\mathbf b\);
- use the Invertible Matrix Theorem to move between different statements about a square matrix.
Day 1: Matrix Operations
Today we learn how to add, scale, multiply and transpose matrices, and which rules of ordinary algebra still hold.
1.1 Notation
If \(A\) is an \(m\times n\) matrix (\(m\) rows, \(n\) columns), the entry in row \(i\) and column \(j\) is written \(a_{ij}\) and is called the \((i,j)\)-entry of \(A\). Each column of \(A\) is a vector in \(\mathbb R^m\). We write the columns as \(\mathbf a_1,\dots,\mathbf a_n\) and the matrix as
\[A=\begin{bmatrix}\mathbf a_1 & \mathbf a_2 & \cdots & \mathbf a_n\end{bmatrix}.\]The entries \(a_{11},a_{22},a_{33},\dots\) form the main diagonal of \(A\). A diagonal matrix is a square matrix whose entries outside the main diagonal are all zero. The \(n\times n\) identity matrix \(I_n\) is the diagonal matrix with 1's on the diagonal. A matrix whose entries are all zero is a zero matrix, written \(0\).
1.2 Sums and scalar multiples
Two matrices are equal if they have the same size and their corresponding entries are equal.
If \(A\) and \(B\) are both \(m\times n\), the sum \(A+B\) is the \(m\times n\) matrix whose entries are the sums of the corresponding entries of \(A\) and \(B\). The sum is defined only when \(A\) and \(B\) have the same size.
If \(r\) is a scalar, the scalar multiple \(rA\) is the matrix whose entries are \(r\) times the entries of \(A\). We write \(-A\) for \((-1)A\) and \(A-B\) for \(A+(-1)B\).
Let
\[A=\begin{bmatrix}4&0&5\\-1&3&2\end{bmatrix},\qquad B=\begin{bmatrix}1&1&1\\3&5&7\end{bmatrix},\qquad C=\begin{bmatrix}2&-3\\0&1\end{bmatrix}.\]Compute, if defined:
Solution
Let \(A\), \(B\), \(C\) be matrices of the same size, and let \(r\), \(s\) be scalars.
- \(A+B=B+A\)
- \((A+B)+C=A+(B+C)\)
- \(A+0=A\)
- \(r(A+B)=rA+rB\)
- \((r+s)A=rA+sA\)
- \(r(sA)=(rs)A\)
Each rule holds because the same rule holds for the columns, which are vectors. So for sums and scalar multiples, matrices behave like numbers.
1.3 Matrix multiplication
When a matrix \(B\) multiplies a vector \(\mathbf x\), it produces the vector \(B\mathbf x\). If we then multiply by a matrix \(A\), we get \(A(B\mathbf x)\). So \(A(B\mathbf x)\) comes from \(\mathbf x\) by a composition of two mappings (Figure 1). We want a single matrix, called \(AB\), that does both steps at once:
\[A(B\mathbf x)=(AB)\mathbf x.\]To find this matrix, let \(A\) be \(m\times n\), let \(B\) be \(n\times p\) with columns \(\mathbf b_1,\dots,\mathbf b_p\), and let \(\mathbf x\) be in \(\mathbb R^p\). Then
\[B\mathbf x=x_1\mathbf b_1+\cdots+x_p\mathbf b_p,\]and since multiplication by \(A\) is linear,
\[A(B\mathbf x)=x_1A\mathbf b_1+\cdots+x_pA\mathbf b_p=\begin{bmatrix}A\mathbf b_1&A\mathbf b_2&\cdots&A\mathbf b_p\end{bmatrix}\mathbf x.\]The matrix in the last expression is the one we are looking for.
If \(A\) is an \(m\times n\) matrix and \(B\) is an \(n\times p\) matrix with columns \(\mathbf b_1,\dots,\mathbf b_p\), then the product \(AB\) is the \(m\times p\) matrix whose columns are \(A\mathbf b_1,\dots,A\mathbf b_p\).
Multiplication of matrices corresponds to composition of linear transformations.
Compute \(AB\), where \(A=\begin{bmatrix}2&3\\1&-5\end{bmatrix}\) and \(B=\begin{bmatrix}4&3&6\\1&-2&3\end{bmatrix}\).
Solution
Write \(B=[\,\mathbf b_1\ \mathbf b_2\ \mathbf b_3\,]\) and multiply \(A\) by each column of \(B\):
\[A\mathbf b_1=\begin{bmatrix}2&3\\1&-5\end{bmatrix}\begin{bmatrix}4\\1\end{bmatrix}=\begin{bmatrix}2(4)+3(1)\\1(4)-5(1)\end{bmatrix}=\begin{bmatrix}11\\-1\end{bmatrix}\] \[A\mathbf b_2=\begin{bmatrix}2&3\\1&-5\end{bmatrix}\begin{bmatrix}3\\-2\end{bmatrix}=\begin{bmatrix}2(3)+3(-2)\\1(3)-5(-2)\end{bmatrix}=\begin{bmatrix}0\\13\end{bmatrix}\] \[A\mathbf b_3=\begin{bmatrix}2&3\\1&-5\end{bmatrix}\begin{bmatrix}6\\3\end{bmatrix}=\begin{bmatrix}2(6)+3(3)\\1(6)-5(3)\end{bmatrix}=\begin{bmatrix}21\\-9\end{bmatrix}\]These three vectors are the columns of the product:
\[AB=\begin{bmatrix}11&0&21\\-1&13&-9\end{bmatrix}.\]Note that each column of \(AB\) is a linear combination of the columns of \(A\), with weights from the corresponding column of \(B\).
The product \(A\mathbf b_1\) is defined only if \(A\) has as many columns as \(\mathbf b_1\) has entries. So the number of columns of \(A\) must match the number of rows of \(B\). The product \(AB\) then has as many rows as \(A\) and as many columns as \(B\).
The inner sizes must match. The outer sizes give the size of \(AB\).
If \(A\) is a \(3\times 5\) matrix and \(B\) is a \(5\times 2\) matrix, what are the sizes of \(AB\) and \(BA\), if they are defined?
Solution
\(A\) has 5 columns and \(B\) has 5 rows, so \(AB\) is defined. Its size is \(3\times 2\) (rows of \(A\), columns of \(B\)).
For \(BA\) we would need the 2 columns of \(B\) to match the 3 rows of \(A\). They do not match, so \(BA\) is not defined.
The row-column rule
The definition is the right way to think about \(AB\). For hand computation of single entries, the next rule is faster.
If \(AB\) is defined, then the entry in row \(i\) and column \(j\) of \(AB\) is the sum of the products of corresponding entries from row \(i\) of \(A\) and column \(j\) of \(B\).
Row \(i\) of \(A\) times column \(j\) of \(B\).
The rule is true because column \(j\) of \(AB\) is \(A\mathbf b_j\), and the \(i\)th entry of \(A\mathbf b_j\) is computed from row \(i\) of \(A\) and the vector \(\mathbf b_j\).
Solution
Row 2 of \(A\) is \((1,\ -5)\) and column 2 of \(B\) is \((3,\ -2)\), so \((AB)_{22}=1(3)+(-5)(-2)=13\).
Both values agree with the matrix found in Example 2.
Self-check only. These exercises are not graded.
1.4 Properties of matrix multiplication
Let \(A\) be \(m\times n\), and let \(B\) and \(C\) have sizes for which the sums and products below are defined.
- \(A(BC)=(AB)C\) (associative law)
- \(A(B+C)=AB+AC\) (left distributive law)
- \((B+C)A=BA+CA\) (right distributive law)
- \(r(AB)=(rA)B=A(rB)\) for any scalar \(r\)
- \(I_mA=A=AI_n\) (identity for matrix multiplication)
Property (a) holds because matrix multiplication is composition of mappings, and composition of functions is associative. Because of (a), we can write \(ABC\) without parentheses. We may group the factors as we like, but we must keep their left-to-right order. The order matters, as the next example shows.
Let \(A=\begin{bmatrix}5&1\\3&-2\end{bmatrix}\) and \(B=\begin{bmatrix}2&0\\4&3\end{bmatrix}\). Show that \(AB\neq BA\).
Solution
We use the row-column rule for each entry.
\[AB=\begin{bmatrix}5(2)+1(4)&5(0)+1(3)\\3(2)-2(4)&3(0)-2(3)\end{bmatrix}=\begin{bmatrix}14&3\\-2&-6\end{bmatrix}\] \[BA=\begin{bmatrix}2(5)+0(3)&2(1)+0(-2)\\4(5)+3(3)&4(1)+3(-2)\end{bmatrix}=\begin{bmatrix}10&2\\29&-2\end{bmatrix}\]The two products are different, so \(A\) and \(B\) do not commute. This is not surprising: the columns of \(AB\) are combinations of the columns of \(A\), while the columns of \(BA\) are built from the columns of \(B\).
- In general, \(AB\neq BA\).
- The cancellation laws do not hold. If \(AB=AC\), it is not true in general that \(B=C\).
- If a product \(AB\) is the zero matrix, you cannot conclude in general that \(A=0\) or \(B=0\).
\(AB\neq BA\) in general.
\(AB=AC\) does not give \(B=C\).
\(AB=0\) does not give \(A=0\) or \(B=0\).
Self-check only. Entries may be typed as integers, decimals (1.5 or 1,5) or simple fractions (3/2).
Let \(A=\begin{bmatrix}3&-6\\-4&8\end{bmatrix}\), \(B=\begin{bmatrix}8&6\\5&7\end{bmatrix}\), \(C=\begin{bmatrix}6&-2\\4&3\end{bmatrix}\). Compute \(AB\) and \(AC\).
So \(AB=AC\) although \(B\neq C\). We cannot cancel \(A\) from both sides.
Self-check only. Many answers are correct; your matrix is checked by computing \(AB\).
Let \(A=\begin{bmatrix}2&-8\\-1&4\end{bmatrix}\). Construct a \(2\times 2\) matrix \(B\) such that \(AB\) is the zero matrix. Use two different nonzero columns for \(B\).
\(AB=[\,A\mathbf b_1\ \ A\mathbf b_2\,]\), so we need \(A\mathbf b_1=\mathbf 0\) and \(A\mathbf b_2=\mathbf 0\). The equation \(A\mathbf x=\mathbf 0\) reads \(2x_1-8x_2=0\) (the second row is a multiple of the first), so \(x_1=4x_2\). Every solution is a multiple of \((4,1)\). Take two different nonzero multiples, for example
\[B=\begin{bmatrix}4&8\\1&2\end{bmatrix}.\]Then \(AB=0\) although \(A\neq 0\) and \(B\neq 0\).
1.5 Powers of a matrix
If \(A\) is an \(n\times n\) matrix and \(k\) is a positive integer, then \(A^k\) is the product of \(k\) copies of \(A\):
\[A^k=\underbrace{A\cdots A}_{k}.\]The vector \(A^k\mathbf x\) is the result of multiplying \(\mathbf x\) by \(A\) repeatedly, \(k\) times. For \(k=0\) nothing should happen to \(\mathbf x\), so \(A^0\) is interpreted as the identity matrix.
1.6 The transpose of a matrix
Given an \(m\times n\) matrix \(A\), the transpose of \(A\) is the \(n\times m\) matrix \(A^T\) whose columns are formed from the corresponding rows of \(A\).
Let
\[A=\begin{bmatrix}a&b\\c&d\end{bmatrix},\qquad B=\begin{bmatrix}-5&2\\1&-3\\0&4\end{bmatrix},\qquad C=\begin{bmatrix}1&1&1&1\\-3&5&-2&7\end{bmatrix}.\]Row 1 of each matrix becomes column 1 of its transpose, row 2 becomes column 2, and so on:
\[A^T=\begin{bmatrix}a&c\\b&d\end{bmatrix},\qquad B^T=\begin{bmatrix}-5&1&0\\2&-3&4\end{bmatrix},\qquad C^T=\begin{bmatrix}1&-3\\1&5\\1&-2\\1&7\end{bmatrix}.\]Let \(A\) and \(B\) be matrices whose sizes are appropriate for the following sums and products.
- \((A^T)^T=A\)
- \((A+B)^T=A^T+B^T\)
- \((rA)^T=rA^T\) for any scalar \(r\)
- \((AB)^T=B^TA^T\)
The transpose of a product is the product of the transposes in the reverse order. Usually \((AB)^T\neq A^TB^T\).
A vector in \(\mathbb R^n\) can be seen as an \(n\times 1\) matrix, so Theorem 3 applies to vectors too. Let
\[A=\begin{bmatrix}1&-3\\-2&4\end{bmatrix},\qquad \mathbf x=\begin{bmatrix}5\\3\end{bmatrix}.\]Solution
Day 2: The Inverse of a Matrix
Today we define the inverse of a square matrix, learn a formula for the \(2\times 2\) case and a row reduction algorithm for the general case.
- The \((i,j)\)-entry of \(AB\) is row \(i\) of \(A\) times column \(j\) of \(B\).
- \(I_n\) is the \(n\times n\) identity matrix: \(I_nA=A\) and \(AI_n=A\) whenever the products are defined. We write \(I\) when the size is clear. The columns of \(I_n\) are \(\mathbf e_1,\dots,\mathbf e_n\).
- In general \(AB\neq BA\), so in a matrix equation we must say on which side we multiply.
2.1 Invertible matrices
For a nonzero number such as 5, the multiplicative inverse \(5^{-1}=1/5\) satisfies \(5^{-1}\cdot 5=1\) and \(5\cdot 5^{-1}=1\). For matrices we ask for both equations, because matrix multiplication is not commutative. We also avoid the notation \(1/A\). The idea works fully only for square matrices.
An \(n\times n\) matrix \(A\) is invertible if there is an \(n\times n\) matrix \(C\) such that
\[CA=I\quad\text{and}\quad AC=I.\]In this case \(C\) is called the inverse of \(A\) and is written \(A^{-1}\). A matrix that is not invertible is called singular, and an invertible matrix is called nonsingular.
The inverse is unique. If \(B\) were another inverse of \(A\), then \(B=BI=B(AC)=(BA)C=IC=C\).
Only square matrices can be invertible. Never write \(\dfrac{1}{A}\).
Let \(A=\begin{bmatrix}2&5\\-3&-7\end{bmatrix}\) and \(C=\begin{bmatrix}-7&-5\\3&2\end{bmatrix}\). Show that \(C=A^{-1}\).
Solution
By the definition we must check both products:
\[AC=\begin{bmatrix}2(-7)+5(3)&2(-5)+5(2)\\-3(-7)-7(3)&-3(-5)-7(2)\end{bmatrix}=\begin{bmatrix}1&0\\0&1\end{bmatrix}\] \[CA=\begin{bmatrix}-7(2)-5(-3)&-7(5)-5(-7)\\3(2)+2(-3)&3(5)+2(-7)\end{bmatrix}=\begin{bmatrix}1&0\\0&1\end{bmatrix}\]Both products are \(I\), so \(A\) is invertible and \(C=A^{-1}\).
2.2 The \(2\times2\) case
Let \(A=\begin{bmatrix}a&b\\c&d\end{bmatrix}\). If \(ad-bc\neq0\), then \(A\) is invertible and
\[A^{-1}=\frac{1}{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}.\]If \(ad-bc=0\), then \(A\) is not invertible.
The number \(ad-bc\) is called the determinant of \(A\), written \(\det A=ad-bc\).
Swap \(a\) and \(d\), change the signs of \(b\) and \(c\), divide by \(\det A\). A \(2\times2\) matrix is invertible exactly when \(\det A\neq0\).
Find the inverse of \(A=\begin{bmatrix}3&4\\5&6\end{bmatrix}\).
Solution
\(\det A=3(6)-4(5)=-2\neq0\), so \(A\) is invertible. By Theorem 4,
\[A^{-1}=\frac{1}{-2}\begin{bmatrix}6&-4\\-5&3\end{bmatrix}=\begin{bmatrix}-3&2\\5/2&-3/2\end{bmatrix}.\]Use determinants to decide which of the following matrices are invertible.
Solution
Figure 2 shows the columns of the matrices in (a) and (c). For a \(2\times2\) matrix, \(ad-bc=0\) happens exactly when one column is a multiple of the other, that is, when both columns lie on one line through the origin. In (c), \((-9,6)=-\tfrac32\,(6,-4)\).
2.3 Solving \(A\mathbf x=\mathbf b\) with an inverse
If \(A\) is an invertible \(n\times n\) matrix, then for each \(\mathbf b\) in \(\mathbb R^n\), the equation \(A\mathbf x=\mathbf b\) has the unique solution \(\mathbf x=A^{-1}\mathbf b\).
Why. The vector \(A^{-1}\mathbf b\) is a solution, because \(A(A^{-1}\mathbf b)=(AA^{-1})\mathbf b=I\mathbf b=\mathbf b\). It is the only one: if \(A\mathbf u=\mathbf b\), multiply both sides on the left by \(A^{-1}\) to get \(A^{-1}A\mathbf u=A^{-1}\mathbf b\), that is, \(\mathbf u=A^{-1}\mathbf b\).
If \(A\) is invertible, then \(A\mathbf x=\mathbf b\) has exactly one solution:
\[\mathbf x=A^{-1}\mathbf b\]Use the inverse of the matrix \(A\) in Example 2 to solve the system
\[\begin{aligned}3x_1+4x_2&=3\\5x_1+6x_2&=7\end{aligned}\]Solution
The system is \(A\mathbf x=\mathbf b\) with \(A=\begin{bmatrix}3&4\\5&6\end{bmatrix}\) and \(\mathbf b=\begin{bmatrix}3\\7\end{bmatrix}\). \(A\) is invertible (Example 2), so by Theorem 5
\[\mathbf x=A^{-1}\mathbf b=\begin{bmatrix}-3&2\\5/2&-3/2\end{bmatrix}\begin{bmatrix}3\\7\end{bmatrix}=\begin{bmatrix}-9+14\\15/2-21/2\end{bmatrix}=\begin{bmatrix}5\\-3\end{bmatrix}.\]Check: \(3(5)+4(-3)=3\) and \(5(5)+6(-3)=7\).
For larger systems, row reduction of \([\,A\mid\mathbf b\,]\) is almost always faster than computing \(A^{-1}\). The formula \(\mathbf x=A^{-1}\mathbf b\) is mainly a tool for reasoning and for the \(2\times2\) case.
Self-check only. Your inverse is checked by computing the product with \(A\).
2.4 Inverses of products and transposes
- If \(A\) is invertible, then \(A^{-1}\) is invertible and \((A^{-1})^{-1}=A\).
- If \(A\) and \(B\) are \(n\times n\) invertible matrices, then so is \(AB\), and \((AB)^{-1}=B^{-1}A^{-1}\).
- If \(A\) is invertible, then so is \(A^T\), and \((A^T)^{-1}=(A^{-1})^T\).
Why (b) holds. To show that \(B^{-1}A^{-1}\) is the inverse of \(AB\), we check the definition:
\[(AB)(B^{-1}A^{-1})=A(BB^{-1})A^{-1}=AIA^{-1}=AA^{-1}=I,\]and in the same way \((B^{-1}A^{-1})(AB)=I\). For (c), use Theorem 3(d): \((A^{-1})^TA^T=(AA^{-1})^T=I^T=I\), and similarly \(A^T(A^{-1})^T=I\).
The inverse of a product is the product of the inverses in the reverse order. The same is true for any number of invertible \(n\times n\) factors.
If \(A\) is an invertible matrix, prove that \(5A\) is an invertible matrix.
Solution
To prove that a matrix is invertible, we show a matrix that satisfies the definition. Since \(A\) is invertible, there is a matrix \(C\) with \(AC=I=CA\). Try \(D=\tfrac15C\). By Theorem 2(d), scalars can be moved through a product:
\[(5A)\left(\tfrac15C\right)=5\cdot\tfrac15\,(AC)=1\cdot I=I,\qquad \left(\tfrac15C\right)(5A)=\tfrac15\cdot5\,(CA)=I.\]So \(5A\) is invertible, and \((5A)^{-1}=\tfrac15A^{-1}\).
2.5 An algorithm for finding \(A^{-1}\)
Suppose first that \(A\) is invertible. The equation \(AA^{-1}=I\) and the definition of the product say that \(A\) times column \(j\) of \(A^{-1}\) is column \(j\) of \(I\). So the columns of \(A^{-1}\) are the solutions of the \(n\) systems
\[A\mathbf x=\mathbf e_1,\qquad A\mathbf x=\mathbf e_2,\qquad\dots,\qquad A\mathbf x=\mathbf e_n.\]All these systems have the same coefficient matrix, so the same row operations solve all of them. We can solve them at the same time by placing all the right sides next to \(A\):
\[[\,A\mid\mathbf e_1\ \mathbf e_2\ \cdots\ \mathbf e_n\,]=[\,A\mid I\,].\]If \(A\) is row equivalent to \(I\), the reduced echelon form of this big matrix is \([\,I\mid X\,]\), and column \(j\) of \(X\) is the solution of \(A\mathbf x=\mathbf e_j\). So \(AX=I\). For square matrices this one equation is enough to conclude that \(A\) is invertible and \(X=A^{-1}\) (this fact is stated on Day 3). If \(A\) is not row equivalent to \(I\), then \(A\) has fewer than \(n\) pivot positions, so \(A\mathbf x=\mathbf 0\) has a nontrivial solution. An invertible matrix cannot have this property, because Theorem 5 gives \(\mathbf x=A^{-1}\mathbf 0=\mathbf 0\) as the only solution.
An \(n\times n\) matrix \(A\) is invertible if and only if \(A\) is row equivalent to \(I_n\). In this case, any sequence of elementary row operations that reduces \(A\) to \(I_n\) also transforms \(I_n\) into \(A^{-1}\).
Row reduce the augmented matrix \([\,A\mid I\,]\). If \(A\) is row equivalent to \(I\), then \([\,A\mid I\,]\) is row equivalent to \([\,I\mid A^{-1}\,]\). Otherwise, \(A\) does not have an inverse.
If the left block cannot be reduced to \(I\), then \(A\) is not invertible.
Find the inverse of \(A=\begin{bmatrix}0&1&2\\1&0&3\\4&-3&8\end{bmatrix}\), if it exists.
Solution
Row reduce \([\,A\mid I\,]\). The first entry of row 1 is 0, so we start with an interchange \(R_1\leftrightarrow R_2\). Then \(R_3\to R_3-4R_1\) clears the first column:
\[\left[\begin{array}{ccc|ccc}0&1&2&1&0&0\\1&0&3&0&1&0\\4&-3&8&0&0&1\end{array}\right] \sim\left[\begin{array}{ccc|ccc}1&0&3&0&1&0\\0&1&2&1&0&0\\4&-3&8&0&0&1\end{array}\right] \sim\left[\begin{array}{ccc|ccc}1&0&3&0&1&0\\0&1&2&1&0&0\\0&-3&-4&0&-4&1\end{array}\right]\]Next \(R_3\to R_3+3R_2\) clears the second column below the pivot, and \(R_3\to\tfrac12R_3\) makes the third pivot equal to 1:
\[\sim\left[\begin{array}{ccc|ccc}1&0&3&0&1&0\\0&1&2&1&0&0\\0&0&2&3&-4&1\end{array}\right] \sim\left[\begin{array}{ccc|ccc}1&0&3&0&1&0\\0&1&2&1&0&0\\0&0&1&3/2&-2&1/2\end{array}\right]\]Finally \(R_1\to R_1-3R_3\) and \(R_2\to R_2-2R_3\) clear the third column above the pivot:
\[\sim\left[\begin{array}{ccc|ccc}1&0&0&-9/2&7&-3/2\\0&1&0&-2&4&-1\\0&0&1&3/2&-2&1/2\end{array}\right]\]The left block is \(I\), so \(A\) is invertible and
\[A^{-1}=\begin{bmatrix}-9/2&7&-3/2\\-2&4&-1\\3/2&-2&1/2\end{bmatrix}.\]Check. It is a good habit to multiply:
\[AA^{-1}=\begin{bmatrix}0&1&2\\1&0&3\\4&-3&8\end{bmatrix}\begin{bmatrix}-9/2&7&-3/2\\-2&4&-1\\3/2&-2&1/2\end{bmatrix}=\begin{bmatrix}1&0&0\\0&1&0\\0&0&1\end{bmatrix}.\]For example, the \((1,1)\)-entry is \(0(-9/2)+1(-2)+2(3/2)=1\) and the \((3,1)\)-entry is \(4(-9/2)-3(-2)+8(3/2)=-18+6+12=0\).
Find the inverse of \(A=\begin{bmatrix}1&-2&-1\\-1&5&6\\5&-4&5\end{bmatrix}\), if it exists.
Solution
Row reduce \([\,A\mid I\,]\) with \(R_2\to R_2+R_1\) and \(R_3\to R_3-5R_1\), then \(R_3\to R_3-2R_2\):
\[\left[\begin{array}{ccc|ccc}1&-2&-1&1&0&0\\-1&5&6&0&1&0\\5&-4&5&0&0&1\end{array}\right] \sim\left[\begin{array}{ccc|ccc}1&-2&-1&1&0&0\\0&3&5&1&1&0\\0&6&10&-5&0&1\end{array}\right]\] \[\sim\left[\begin{array}{ccc|ccc}1&-2&-1&1&0&0\\0&3&5&1&1&0\\0&0&0&-7&-2&1\end{array}\right]\]The left block now has a row of zeros. It has only two pivot positions, so it cannot be reduced to \(I\). We stop here: \(A\) does not have an inverse.
Self-check only. Your vector is checked by computing \(A\mathbf x\).
Let \(A=\begin{bmatrix}-2&-7&-9\\2&5&6\\1&3&4\end{bmatrix}\). Find the third column of \(A^{-1}\) without computing the other columns.
The third column of \(A^{-1}\) is the solution of \(A\mathbf x=\mathbf e_3\). Row reduce \([\,A\mid\mathbf e_3\,]\). Start with \(R_1\leftrightarrow R_3\), then \(R_2\to R_2-2R_1\) and \(R_3\to R_3+2R_1\):
\[\left[\begin{array}{ccc|c}-2&-7&-9&0\\2&5&6&0\\1&3&4&1\end{array}\right] \sim\left[\begin{array}{ccc|c}1&3&4&1\\2&5&6&0\\-2&-7&-9&0\end{array}\right] \sim\left[\begin{array}{ccc|c}1&3&4&1\\0&-1&-2&-2\\0&-1&-1&2\end{array}\right]\]Then \(R_3\to R_3-R_2\) and \(R_2\to -R_2\); after that \(R_2\to R_2-2R_3\), \(R_1\to R_1-4R_3\) and \(R_1\to R_1-3R_2\):
\[\sim\left[\begin{array}{ccc|c}1&3&4&1\\0&1&2&2\\0&0&1&4\end{array}\right] \sim\left[\begin{array}{ccc|c}1&0&0&3\\0&1&0&-6\\0&0&1&4\end{array}\right]\]The third column of \(A^{-1}\) is \((3,-6,4)\). Check: \(-2(3)-7(-6)-9(4)=0\), \(\ 2(3)+5(-6)+6(4)=0\), \(\ 1(3)+3(-6)+4(4)=1\).
Day 3: Characterizations of Invertible Matrices
Today we connect invertibility with the ideas of the earlier weeks: pivots, solutions of \(A\mathbf x=\mathbf b\), linear independence, span, and linear transformations.
- \(A\) is invertible if there is a matrix \(C\) with \(CA=I\) and \(AC=I\). For \(A=\begin{bmatrix}a&b\\c&d\end{bmatrix}\) with \(ad-bc\neq0\): \(A^{-1}=\dfrac{1}{ad-bc}\begin{bmatrix}d&-b\\-c&a\end{bmatrix}\). If \(ad-bc=0\), \(A\) is not invertible.
- If \(A\) is invertible, \(A\mathbf x=\mathbf b\) has the unique solution \(\mathbf x=A^{-1}\mathbf b\). \(A\) is invertible exactly when \(A\) is row equivalent to \(I_n\).
- The columns of \(A\) are linearly independent exactly when \(A\mathbf x=\mathbf 0\) has only the trivial solution, that is, when every column of \(A\) is a pivot column.
- The columns of an \(m\times n\) matrix \(A\) span \(\mathbb R^m\) exactly when \(A\mathbf x=\mathbf b\) has a solution for every \(\mathbf b\), that is, when \(A\) has a pivot position in every row.
- The transformation \(\mathbf x\mapsto A\mathbf x\) is one-to-one exactly when the columns of \(A\) are linearly independent, and it maps \(\mathbb R^n\) onto \(\mathbb R^m\) exactly when the columns of \(A\) span \(\mathbb R^m\).
3.1 The Invertible Matrix Theorem
Let \(A\) be a square \(n\times n\) matrix. Then the following statements are equivalent. That is, for a given \(A\), the statements are either all true or all false.
- \(A\) is an invertible matrix.
- \(A\) is row equivalent to the \(n\times n\) identity matrix.
- \(A\) has \(n\) pivot positions.
- The equation \(A\mathbf x=\mathbf 0\) has only the trivial solution.
- The columns of \(A\) form a linearly independent set.
- The linear transformation \(\mathbf x\mapsto A\mathbf x\) is one-to-one.
- The equation \(A\mathbf x=\mathbf b\) has at least one solution for each \(\mathbf b\) in \(\mathbb R^n\).
- The columns of \(A\) span \(\mathbb R^n\).
- The linear transformation \(\mathbf x\mapsto A\mathbf x\) maps \(\mathbb R^n\) onto \(\mathbb R^n\).
- There is an \(n\times n\) matrix \(C\) such that \(CA=I\).
- There is an \(n\times n\) matrix \(D\) such that \(AD=I\).
- \(A^T\) is an invertible matrix.
For a square \(n\times n\) matrix \(A\), all true or all false:
\(A\) is invertible
\(\iff\ A\sim I_n\ \iff\ A\) has \(n\) pivot positions
\(\iff\ A\mathbf x=\mathbf 0\) has only the trivial solution
\(\iff\) the columns of \(A\) are linearly independent
\(\iff\ A\mathbf x=\mathbf b\) has a solution for every \(\mathbf b\)
\(\iff\) the columns of \(A\) span \(\mathbb R^n\)
How the statements are linked. The proof is a chain of implications that closes into a circle:
\[(a)\Rightarrow(j)\Rightarrow(d)\Rightarrow(c)\Rightarrow(b)\Rightarrow(a).\]- (a) \(\Rightarrow\) (j): take \(C=A^{-1}\).
- (j) \(\Rightarrow\) (d): if \(A\mathbf x=\mathbf 0\), multiply by \(C\) on the left: \(\mathbf x=CA\mathbf x=C\mathbf 0=\mathbf 0\).
- (d) \(\Rightarrow\) (c): only the trivial solution means no free variables, so all \(n\) columns are pivot columns.
- (c) \(\Rightarrow\) (b): \(n\) pivots in a square matrix must lie on the main diagonal, so the reduced echelon form of \(A\) is \(I_n\).
- (b) \(\Rightarrow\) (a): this is Theorem 7.
The other statements are attached to this circle. (a) \(\Rightarrow\) (k) with \(D=A^{-1}\). (k) \(\Rightarrow\) (g), because \(\mathbf x=D\mathbf b\) solves \(A\mathbf x=\mathbf b\): \(A(D\mathbf b)=I\mathbf b=\mathbf b\). (g) \(\Rightarrow\) (c), because a pivot in every row of a square matrix gives \(n\) pivots. Statements (g), (h), (i) say the same thing for any matrix, and so do (d), (e), (f). Finally (a) and (l) are linked by Theorem 6(c).
One useful consequence is that for square matrices a one-sided inverse is already an inverse.
\(A\), \(B\) square and \(AB=I\) \(\ \Longrightarrow\ \) \(A\) and \(B\) are both invertible, \(B=A^{-1}\) and \(A=B^{-1}\).
The theorem divides all \(n\times n\) matrices into two classes: invertible and singular. Each statement describes every invertible matrix, and the negation of each statement describes every singular matrix. For example, an \(n\times n\) singular matrix is not row equivalent to \(I_n\), has fewer than \(n\) pivot positions, and has linearly dependent columns.
Use the Invertible Matrix Theorem to decide if \(A\) is invertible:
\[A=\begin{bmatrix}1&0&-2\\3&1&-2\\-5&-1&9\end{bmatrix}\]Solution
We only need an echelon form, not the inverse. Use \(R_2\to R_2-3R_1\) and \(R_3\to R_3+5R_1\), then \(R_3\to R_3+R_2\):
\[A\sim\begin{bmatrix}1&0&-2\\0&1&4\\0&-1&-1\end{bmatrix}\sim\begin{bmatrix}1&0&-2\\0&1&4\\0&0&3\end{bmatrix}\]\(A\) has three pivot positions, so \(A\) is invertible by statement (c).
Determine if \(A=\begin{bmatrix}2&3&4\\2&3&4\\2&3&4\end{bmatrix}\) is invertible.
Solution
Columns 2 and 3 are multiples of column 1: \(\mathbf a_2=\tfrac32\mathbf a_1\) and \(\mathbf a_3=2\mathbf a_1\). So the columns of \(A\) are linearly dependent, statement (e) is false, and \(A\) is not invertible.
The Invertible Matrix Theorem applies only to square matrices. For example, if the columns of a \(4\times3\) matrix are linearly independent, the theorem tells us nothing about the existence of solutions of \(A\mathbf x=\mathbf b\).
Self-check only. These exercises are not graded.
Decide for each matrix whether it is invertible. Use as few calculations as possible.
Suppose \(A\) and \(B\) are \(n\times n\) matrices and the equation \(AB\mathbf x=\mathbf 0\) has a nontrivial solution. What can you say about the matrix \(AB\)?
Solution
\(AB\) is an \(n\times n\) matrix, so we can apply the theorem to \(AB\) in place of \(A\). Statement (d) for \(AB\) says "\(AB\mathbf x=\mathbf 0\) has only the trivial solution", and this is false. So all the statements are false for \(AB\). In particular, \(AB\) is not invertible.
3.2 Invertible linear transformations
Matrix multiplication corresponds to composition of linear transformations. When \(A\) is invertible, the equation \(A^{-1}A\mathbf x=\mathbf x\) says that multiplication by \(A^{-1}\) brings \(A\mathbf x\) back to \(\mathbf x\) (Figure 3).
A linear transformation \(T:\mathbb R^n\to\mathbb R^n\) is invertible if there is a function \(S:\mathbb R^n\to\mathbb R^n\) such that
\[S(T(\mathbf x))=\mathbf x\ \text{ for all }\mathbf x\text{ in }\mathbb R^n\qquad\text{and}\qquad T(S(\mathbf x))=\mathbf x\ \text{ for all }\mathbf x\text{ in }\mathbb R^n.\]\(S\) is called the inverse of \(T\) and is written \(T^{-1}\).
Let \(T:\mathbb R^n\to\mathbb R^n\) be a linear transformation and let \(A\) be the standard matrix of \(T\). Then \(T\) is invertible if and only if \(A\) is an invertible matrix. In that case, \(S(\mathbf x)=A^{-1}\mathbf x\) is the unique function that satisfies the two equations above.
\(T(\mathbf x)=A\mathbf x\) is invertible \(\iff\) \(A\) is invertible, and then \(T^{-1}(\mathbf x)=A^{-1}\mathbf x\).
What can you say about a one-to-one linear transformation \(T\) from \(\mathbb R^n\) into \(\mathbb R^n\)?
Solution
Let \(A\) be the standard matrix of \(T\). It is \(n\times n\). Since \(T\) is one-to-one, the columns of \(A\) are linearly independent. This is statement (e), so all statements of the Invertible Matrix Theorem hold: \(A\) is invertible, and by statement (i) \(T\) maps \(\mathbb R^n\) onto \(\mathbb R^n\). By Theorem 9, \(T\) is invertible.
Let \(T:\mathbb R^2\to\mathbb R^2\) be given by \(T(x_1,x_2)=(-9x_1+7x_2,\ 4x_1-3x_2)\). Show that \(T\) is invertible and find a formula for \(T^{-1}\).
Solution
The standard matrix of \(T\) has the coefficients of \(x_1\) in its first column and those of \(x_2\) in its second column:
\[A=\begin{bmatrix}-9&7\\4&-3\end{bmatrix},\qquad \det A=(-9)(-3)-7(4)=27-28=-1\neq0.\]So \(A\) is invertible, and \(T\) is invertible by Theorem 9. By the \(2\times2\) formula,
\[A^{-1}=\frac{1}{-1}\begin{bmatrix}-3&-7\\-4&-9\end{bmatrix}=\begin{bmatrix}3&7\\4&9\end{bmatrix},\]and therefore \(T^{-1}(x_1,x_2)=(3x_1+7x_2,\ 4x_1+9x_2)\).
Check with one vector: \(T(1,0)=(-9,4)\) and \(T^{-1}(-9,4)=(-27+28,\ -36+36)=(1,0)\).
Self-check only. All matrices are \(n\times n\). Mark an implication True if statement 2 always follows when statement 1 is true.
Homework 5
- Fill in your name and surname below before you save the PDF.
- Show your work. An answer without the steps that lead to it does not receive full credit.
- Every question has two answer modes: Type and Write by hand. In Write by hand mode the text boxes are replaced by a writing area where you can write with a pen, your finger or the mouse. For each question, only the answer of the selected mode goes into the PDF. Changing the mode does not delete what you entered in the other mode.
- In Type mode, write a matrix row by row, for example
[1 2; 3 4], when there is no grid for it. In the row reduction tool, write the row operation (for exampleR2 -> R2 - 3R1) and then the new matrix. - To submit: press "Save homework as PDF", choose "Save as PDF" in the print dialog, and upload the file to Canvas.
- Your answers are saved automatically in this browser. If you open the page on a different computer or browser, they will not be there.
Question 115 points
Let
\[A=\begin{bmatrix}2&0&-1\\4&-3&2\end{bmatrix},\quad B=\begin{bmatrix}7&-5&1\\1&-4&-3\end{bmatrix},\quad C=\begin{bmatrix}1&2\\-2&1\end{bmatrix},\quad E=\begin{bmatrix}-5\\3\end{bmatrix}.\]Compute each matrix sum or product if it is defined. If an expression is undefined, explain why.
Question 215 points
Let
\[A=\begin{bmatrix}1&1&1\\1&2&3\\1&4&5\end{bmatrix}\qquad\text{and}\qquad D=\begin{bmatrix}2&0&0\\0&3&0\\0&0&5\end{bmatrix}.\]Question 320 points
Let
\[A=\begin{bmatrix}1&2\\5&12\end{bmatrix},\quad \mathbf b_1=\begin{bmatrix}-1\\3\end{bmatrix},\quad \mathbf b_2=\begin{bmatrix}1\\-5\end{bmatrix}.\]Read the two solutions from your last matrix and compare them with part (a):
Question 420 points
Find the inverses of the following matrices, if they exist. Use the algorithm of Day 2 (row reduce \([\,A\mid I\,]\)). If the inverse exists, verify your answer by computing one row of \(AA^{-1}\).
Conclusion and verification:
Conclusion and verification:
Question 515 points
Determine which of the following matrices are invertible. Use as few calculations as possible, and do not compute any inverse. Justify each answer with a statement of the Invertible Matrix Theorem.
Decision and justification:
Decision and justification:
Question 615 points
In this question all matrices are \(n\times n\). Each part is an implication of the form "If statement 1, then statement 2". Mark it True if statement 2 always follows when statement 1 is true, and False otherwise. Justify each answer: for True, name the statements of the Invertible Matrix Theorem or the theorems you use; for False, give a specific counterexample.