Vector Spaces, Subspaces, Null and Column Spaces, Bases
Vector spaces and subspaces · Null spaces, column spaces, row spaces, and linear transformations · Linearly independent sets; bases
By the end of this week you will be able to:
- work with matrices and functions as vectors, and name the zero vector of each space;
- decide whether a given set is a subspace, and give a specific counterexample when it is not;
- find a spanning set for \(\operatorname{Nul} A\) and decide whether a vector lies in \(\operatorname{Nul} A\) or in \(\operatorname{Col} A\);
- find the kernel and the range of a linear transformation between vector spaces;
- decide whether a set is a basis, and find bases for \(\operatorname{Nul} A\), \(\operatorname{Col} A\) and \(\operatorname{Row} A\).
Day 1: Vector Spaces and Subspaces
Today we see that matrices and functions can be added and scaled just like vectors in \(\mathbb R^n\), and we learn a quick test for deciding whether a set is a subspace.
- Vectors in \(\mathbb R^n\) are added and scaled entry by entry: \((1,2,3)+(4,0,-1)=(5,2,2)\) and \(3(1,2,3)=(3,6,9)\).
- A linear combination of \(\mathbf v_1,\dots,\mathbf v_p\) is a vector \(c_1\mathbf v_1+\cdots+c_p\mathbf v_p\) with scalars \(c_1,\dots,c_p\).
- \(\operatorname{Span}\{\mathbf v_1,\dots,\mathbf v_p\}\) is the set of all linear combinations of \(\mathbf v_1,\dots,\mathbf v_p\).
- A vector \(\mathbf y\) is in \(\operatorname{Span}\{\mathbf v_1,\dots,\mathbf v_p\}\) exactly when the system with augmented matrix \([\,\mathbf v_1\ \cdots\ \mathbf v_p \mid \mathbf y\,]\) is consistent, that is, when an echelon form has no row \([\,0\ \cdots\ 0 \mid b\,]\) with \(b\neq 0\).
1.1 Vector spaces
Everything we did with vectors in \(\mathbb R^n\) used only a few rules for addition and scalar multiplication. Many other sets of objects follow the same rules. Such a set is called a vector space, and its elements are called vectors, even when they are matrices or functions.
A vector space is a nonempty set \(V\) of objects, called vectors, with two operations, addition and multiplication by scalars (real numbers), such that the following ten axioms hold for all \(\mathbf u,\mathbf v,\mathbf w\) in \(V\) and all scalars \(c,d\).
- \(\mathbf u+\mathbf v\) is in \(V\).
- \(\mathbf u+\mathbf v=\mathbf v+\mathbf u\).
- \((\mathbf u+\mathbf v)+\mathbf w=\mathbf u+(\mathbf v+\mathbf w)\).
- There is a zero vector \(\mathbf 0\) in \(V\) with \(\mathbf u+\mathbf 0=\mathbf u\).
- For each \(\mathbf u\) there is a vector \(-\mathbf u\) in \(V\) with \(\mathbf u+(-\mathbf u)=\mathbf 0\).
- \(c\mathbf u\) is in \(V\).
- \(c(\mathbf u+\mathbf v)=c\mathbf u+c\mathbf v\).
- \((c+d)\mathbf u=c\mathbf u+d\mathbf u\).
- \(c(d\mathbf u)=(cd)\mathbf u\).
- \(1\mathbf u=\mathbf u\).
Axioms 1 and 6 say that the two operations never take you outside \(V\). The other axioms are the usual rules of algebra. From the axioms one gets three facts that we use all the time: for every \(\mathbf u\) in \(V\) and every scalar \(c\),
\[0\mathbf u=\mathbf 0,\qquad c\mathbf 0=\mathbf 0,\qquad -\mathbf u=(-1)\mathbf u.\]The zero vector is \((0,\dots,0)\) in \(\mathbb R^n\), the zero matrix in \(M_{m\times n}\), the function that is identically zero in the space of functions, and the zero signal \((\dots,0,0,0,\dots)\) in \(\mathbb S\). The negative of a vector is \((-1)\) times it.
1.2 Subspaces
Most vector spaces we meet sit inside a larger one, for example a plane through the origin inside \(\mathbb R^3\). For such a subset we only need to check three properties. The other axioms are automatic, because they already hold in the larger space.
A subspace of a vector space \(V\) is a subset \(H\) of \(V\) with three properties:
Every subspace is itself a vector space, with the operations of \(V\).
- Check whether \(\mathbf 0\) is in \(H\). If it is not, stop: \(H\) is not a subspace.
- Take two general elements \(\mathbf u,\mathbf v\) of \(H\) and check whether \(\mathbf u+\mathbf v\) satisfies the condition that defines \(H\).
- Take a general \(\mathbf u\) in \(H\) and a general scalar \(c\) and check whether \(c\mathbf u\) satisfies the condition.
To show that \(H\) is not a subspace, one specific counterexample is enough: a specific pair \(\mathbf u,\mathbf v\) in \(H\) with \(\mathbf u+\mathbf v\) not in \(H\), or a specific \(\mathbf u\) and \(c\) with \(c\mathbf u\) not in \(H\).
Let \(H=\left\{\begin{bmatrix} s\\ t\\ 0\end{bmatrix} : s,t \text{ real}\right\}\). Is \(H\) a subspace of \(\mathbb R^3\)?
Solution
The condition for a vector to be in \(H\) is: the third entry is 0.
So \(H\) is a subspace of \(\mathbb R^3\). Note that \(\mathbb R^2\) itself is not a subspace of \(\mathbb R^3\): vectors in \(\mathbb R^2\) have two entries, so \(\mathbb R^2\) is not even a subset of \(\mathbb R^3\). The set \(H\) only looks like \(\mathbb R^2\).
Decide whether each line is a subspace of \(\mathbb R^2\).
Solution
Both sets contain \(\mathbf 0\). The zero test is necessary, but it is not enough. See Figure 2.
Self-check only. These exercises are not graded.
For each set, choose whether it is a subspace of the given space.
Self-check only. Many answers are possible; the check tests whether your vectors really are counterexamples. Entries may be typed as integers, decimals (1.5 or 1,5) or simple fractions (3/2).
1.3 The subspace spanned by a set
The most common way to describe a subspace is as a span. This also gives the fastest way to prove that a set is a subspace.
We call \(\operatorname{Span}\{\mathbf v_1,\dots,\mathbf v_p\}\) the subspace spanned (or generated) by \(\mathbf v_1,\dots,\mathbf v_p\). If \(H=\operatorname{Span}\{\mathbf v_1,\dots,\mathbf v_p\}\), the set \(\{\mathbf v_1,\dots,\mathbf v_p\}\) is a spanning set for \(H\). In \(\mathbb R^3\), the span of one nonzero vector is a line through the origin, and the span of two vectors that are not multiples of each other is a plane through the origin (Figure 3).
Let \(H\) be the set of all vectors \((a-3b,\ b-a,\ a,\ b)\), where \(a\) and \(b\) are any real numbers. Show that \(H\) is a subspace of \(\mathbb R^4\).
Solution
Write a general vector of \(H\) as a column and split it according to the parameters \(a\) and \(b\):
\[\begin{bmatrix} a-3b\\ b-a\\ a\\ b\end{bmatrix}=a\begin{bmatrix} 1\\ -1\\ 1\\ 0\end{bmatrix}+b\begin{bmatrix} -3\\ 1\\ 0\\ 1\end{bmatrix}=a\mathbf v_1+b\mathbf v_2.\]So \(H=\operatorname{Span}\{\mathbf v_1,\mathbf v_2\}\), and \(H\) is a subspace of \(\mathbb R^4\) by Theorem 1. No property had to be checked by hand.
For which values of \(h\) is \(\mathbf y\) in the subspace of \(\mathbb R^3\) spanned by \(\mathbf v_1,\mathbf v_2,\mathbf v_3\)?
\[\mathbf v_1=\begin{bmatrix} 1\\ -1\\ -2\end{bmatrix},\quad \mathbf v_2=\begin{bmatrix} 5\\ -4\\ -7\end{bmatrix},\quad \mathbf v_3=\begin{bmatrix} -3\\ 1\\ 0\end{bmatrix},\quad \mathbf y=\begin{bmatrix} -4\\ 3\\ h\end{bmatrix}.\]Solution
\(\mathbf y\) is in the subspace exactly when \(x_1\mathbf v_1+x_2\mathbf v_2+x_3\mathbf v_3=\mathbf y\) is consistent. Row reduce the augmented matrix with \(R_2\to R_2+R_1\) and \(R_3\to R_3+2R_1\), then \(R_3\to R_3-3R_2\):
\[\left[\begin{array}{ccc|c} 1 & 5 & -3 & -4\\ -1 & -4 & 1 & 3\\ -2 & -7 & 0 & h\end{array}\right]\sim \left[\begin{array}{ccc|c} 1 & 5 & -3 & -4\\ 0 & 1 & -2 & -1\\ 0 & 3 & -6 & h-8\end{array}\right]\sim \left[\begin{array}{ccc|c} 1 & 5 & -3 & -4\\ 0 & 1 & -2 & -1\\ 0 & 0 & 0 & h-5\end{array}\right].\]The system is consistent exactly when \(h-5=0\). So \(\mathbf y\) is in the subspace if and only if \(h=5\).
Self-check only. Several answers are correct; the check tests whether your two vectors lie in \(H\) and span it.
Let \(H\) be the set of all vectors \((a+2b,\ -a,\ 3b,\ a-b)\) with \(a,b\) real. Find vectors \(\mathbf v_1,\mathbf v_2\) with \(H=\operatorname{Span}\{\mathbf v_1,\mathbf v_2\}\). Enter them as the columns of the grid.
Split by the parameters:
\[\begin{bmatrix} a+2b\\ -a\\ 3b\\ a-b\end{bmatrix}=a\begin{bmatrix} 1\\ -1\\ 0\\ 1\end{bmatrix}+b\begin{bmatrix} 2\\ 0\\ 3\\ -1\end{bmatrix}.\]So \(\mathbf v_1=(1,-1,0,1)\) and \(\mathbf v_2=(2,0,3,-1)\) work. Any two vectors of \(H\) that are not multiples of each other also span \(H\), for example \(\mathbf v_1+\mathbf v_2\) and \(\mathbf v_1-\mathbf v_2\).
Self-check only.
Let \(\mathbf v_1=(1,2,-1)\), \(\mathbf v_2=(0,1,3)\) and \(\mathbf y=(2,5,h)\). Find the value of \(h\) for which \(\mathbf y\) is in \(\operatorname{Span}\{\mathbf v_1,\mathbf v_2\}\), and the weights with \(\mathbf y=c_1\mathbf v_1+c_2\mathbf v_2\).
We need \(c_1(1,2,-1)+c_2(0,1,3)=(2,5,h)\). The first entry gives \(c_1=2\). The second gives \(2c_1+c_2=5\), so \(c_2=1\). The third entry must then be \(-c_1+3c_2=-2+3=1\). So \(h=1\).
With row reduction: \(\left[\begin{array}{cc|c} 1 & 0 & 2\\ 2 & 1 & 5\\ -1 & 3 & h\end{array}\right]\sim\left[\begin{array}{cc|c} 1 & 0 & 2\\ 0 & 1 & 1\\ 0 & 3 & h+2\end{array}\right]\sim\left[\begin{array}{cc|c} 1 & 0 & 2\\ 0 & 1 & 1\\ 0 & 0 & h-1\end{array}\right]\), consistent exactly when \(h=1\).
Day 2: Null Spaces, Column Spaces, Row Spaces, and Linear Transformations
Today we meet the subspaces that come with every matrix, \(\operatorname{Nul} A\), \(\operatorname{Col} A\) and \(\operatorname{Row} A\), and the kernel and range of a linear transformation.
- A subspace of \(V\) is a subset that contains \(\mathbf 0\) and is closed under addition and scalar multiplication. A span \(\operatorname{Span}\{\mathbf v_1,\dots,\mathbf v_p\}\) is always a subspace.
- If \(A=[\,\mathbf a_1\ \cdots\ \mathbf a_n\,]\) and \(\mathbf x=(x_1,\dots,x_n)\), then \(A\mathbf x=x_1\mathbf a_1+\cdots+x_n\mathbf a_n\), a linear combination of the columns of \(A\).
- To solve \(A\mathbf x=\mathbf 0\), row reduce \([\,A\mid\mathbf 0\,]\) to reduced echelon form and write the basic variables in terms of the free variables.
- \(A\mathbf x=\mathbf b\) is consistent exactly when an echelon form of \([\,A\mid\mathbf b\,]\) has no row \([\,0\ \cdots\ 0\mid c\,]\) with \(c\neq 0\).
2.1 The null space of a matrix
Let \(A=\begin{bmatrix} 1 & -3 & -2\\ -5 & 9 & 1\end{bmatrix}\) and \(\mathbf u=\begin{bmatrix} 5\\ 3\\ -2\end{bmatrix}\). Is \(\mathbf u\) in \(\operatorname{Nul} A\)?
Solution
We only need to compute \(A\mathbf u\):
\[A\mathbf u=\begin{bmatrix} 5-9+4\\ -25+27-2\end{bmatrix}=\begin{bmatrix} 0\\ 0\end{bmatrix}.\]So \(\mathbf u\) is in \(\operatorname{Nul} A\). Testing membership in a null space takes one multiplication.
Let \(H\) be the set of all vectors \((a,b,c,d)\) in \(\mathbb R^4\) with \(a-2b+5c=d\) and \(c-a=b\). Show that \(H\) is a subspace of \(\mathbb R^4\).
Solution
Move every variable to the left side:
\[\begin{aligned} a-2b+5c-d&=0\\ -a-b+c\phantom{{}-d}&=0\end{aligned}\qquad\text{so}\qquad H=\operatorname{Nul}\begin{bmatrix} 1 & -2 & 5 & -1\\ -1 & -1 & 1 & 0\end{bmatrix}.\]By Theorem 2, \(H\) is a subspace of \(\mathbb R^4\). The equations must be homogeneous: the solutions of \(x_1+x_2=1\), for example, do not include \(\mathbf 0\).
The definition of \(\operatorname{Nul} A\) is implicit: it gives a condition to check, not a list of vectors. Solving \(A\mathbf x=\mathbf 0\) turns it into an explicit description, a spanning set.
Find a spanning set for the null space of \(A=\begin{bmatrix} 1 & 2 & 0 & -1\\ 2 & 4 & 1 & 1\\ -1 & -2 & 2 & 7\end{bmatrix}\).
Solution
Row reduce \([\,A\mid\mathbf 0\,]\): \(R_2\to R_2-2R_1\), \(R_3\to R_3+R_1\), then \(R_3\to R_3-2R_2\).
\[\left[\begin{array}{cccc|c} 1 & 2 & 0 & -1 & 0\\ 2 & 4 & 1 & 1 & 0\\ -1 & -2 & 2 & 7 & 0\end{array}\right]\sim \left[\begin{array}{cccc|c} 1 & 2 & 0 & -1 & 0\\ 0 & 0 & 1 & 3 & 0\\ 0 & 0 & 2 & 6 & 0\end{array}\right]\sim \left[\begin{array}{cccc|c} 1 & 2 & 0 & -1 & 0\\ 0 & 0 & 1 & 3 & 0\\ 0 & 0 & 0 & 0 & 0\end{array}\right].\]The pivots are in columns 1 and 3, so \(x_2\) and \(x_4\) are free, and \(x_1=-2x_2+x_4\), \(x_3=-3x_4\). Write the general solution as a combination with the free variables as weights:
\[\begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4\end{bmatrix}=\begin{bmatrix} -2x_2+x_4\\ x_2\\ -3x_4\\ x_4\end{bmatrix}=x_2\begin{bmatrix} -2\\ 1\\ 0\\ 0\end{bmatrix}+x_4\begin{bmatrix} 1\\ 0\\ -3\\ 1\end{bmatrix}=x_2\mathbf u+x_4\mathbf v.\]So \(\operatorname{Nul} A=\operatorname{Span}\{\mathbf u,\mathbf v\}\). Check: \(A\mathbf u=(-2+2,\,-4+4,\,2-2)=\mathbf 0\) and \(A\mathbf v=(1-1,\,2-3+1,\,-1-6+7)=\mathbf 0\).
Self-check only. Many answers are possible; the check computes with your vectors.
Let \(A=\begin{bmatrix} 1 & 2 & -1\\ 0 & 1 & 1\\ 2 & 5 & -1\end{bmatrix}\).
Self-check only. The check tests that both vectors are in \(\operatorname{Nul} A\) and that together they span it.
Find a spanning set \(\{\mathbf u,\mathbf v\}\) for \(\operatorname{Nul} A\), where
\[A=\begin{bmatrix} 1 & 0 & 2 & 0 & -1\\ 2 & 1 & 3 & 0 & 1\\ -1 & 3 & -5 & 1 & 12\end{bmatrix}.\]Row reduce \([\,A\mid\mathbf 0\,]\). \(R_2\to R_2-2R_1\) and \(R_3\to R_3+R_1\) give rows \((0,1,-1,0,3)\) and \((0,3,-3,1,11)\). Then \(R_3\to R_3-3R_2\) gives \((0,0,0,1,2)\):
\[\left[\begin{array}{ccccc|c} 1 & 0 & 2 & 0 & -1 & 0\\ 0 & 1 & -1 & 0 & 3 & 0\\ 0 & 0 & 0 & 1 & 2 & 0\end{array}\right].\]This is already reduced. Free variables: \(x_3,x_5\). Then \(x_1=-2x_3+x_5\), \(x_2=x_3-3x_5\), \(x_4=-2x_5\), and
\[\mathbf x=x_3\begin{bmatrix} -2\\ 1\\ 1\\ 0\\ 0\end{bmatrix}+x_5\begin{bmatrix} 1\\ -3\\ 0\\ -2\\ 1\end{bmatrix}.\]2.2 The column space of a matrix
Unlike the null space, \(\operatorname{Col} A\) is defined explicitly: the columns are given, and every other vector of \(\operatorname{Col} A\) is built from them.
Find a matrix \(A\) with \(W=\operatorname{Col} A\), where \(W=\{(6a-b,\ a+b,\ -7a) : a,b \text{ real}\}\).
Solution
\[\begin{bmatrix} 6a-b\\ a+b\\ -7a\end{bmatrix}=a\begin{bmatrix} 6\\ 1\\ -7\end{bmatrix}+b\begin{bmatrix} -1\\ 1\\ 0\end{bmatrix},\qquad\text{so}\qquad A=\begin{bmatrix} 6 & -1\\ 1 & 1\\ -7 & 0\end{bmatrix}.\]The vectors of the spanning set become the columns of \(A\). Then \(W=\operatorname{Col} A\), and \(W\) is a subspace of \(\mathbb R^3\).
Let \(A=\begin{bmatrix} 1 & 3\\ 2 & 1\\ -1 & 2\end{bmatrix}\). Describe \(\operatorname{Col} A\) by an equation, and decide whether \(\mathbf b=(5,0,5)\) and \(\mathbf w=(1,1,1)\) are in \(\operatorname{Col} A\).
Solution
Row reduce \([\,A\mid\mathbf b\,]\) for a general \(\mathbf b=(b_1,b_2,b_3)\): \(R_2\to R_2-2R_1\), \(R_3\to R_3+R_1\), then \(R_3\to R_3+R_2\).
\[\left[\begin{array}{cc|c} 1 & 3 & b_1\\ 2 & 1 & b_2\\ -1 & 2 & b_3\end{array}\right]\sim \left[\begin{array}{cc|c} 1 & 3 & b_1\\ 0 & -5 & b_2-2b_1\\ 0 & 5 & b_3+b_1\end{array}\right]\sim \left[\begin{array}{cc|c} 1 & 3 & b_1\\ 0 & -5 & b_2-2b_1\\ 0 & 0 & b_2+b_3-b_1\end{array}\right].\]The system is consistent exactly when \(b_2+b_3-b_1=0\). So \(\operatorname{Col} A\) is the plane \(b_1=b_2+b_3\) in \(\mathbb R^3\).
For the matrix \(A=\begin{bmatrix} 1 & 2\\ 2 & 4\end{bmatrix}\) of Example 4, the second column is twice the first, so \(\operatorname{Col} A=\operatorname{Span}\{(1,2)\}\), another line through the origin. Figure 4 shows both subspaces. They are different lines.
2.3 The row space
2.4 Comparing \(\operatorname{Nul} A\) and \(\operatorname{Col} A\)
Let \(A=\begin{bmatrix} 2 & 4 & -2 & 1\\ -2 & -5 & 7 & 3\\ 3 & 7 & -8 & 6\end{bmatrix}\), \(\mathbf u=\begin{bmatrix} 3\\ -2\\ -1\\ 0\end{bmatrix}\), \(\mathbf v=\begin{bmatrix} 3\\ -1\\ 3\end{bmatrix}\).
Solution
| \(\operatorname{Nul} A\) | \(\operatorname{Col} A\) |
|---|---|
| a subspace of \(\mathbb R^n\) | a subspace of \(\mathbb R^m\) |
| implicit: given by the condition \(A\mathbf x=\mathbf 0\) | explicit: built from the columns of \(A\) |
| finding vectors needs row operations on \([\,A\mid\mathbf 0\,]\) | finding vectors is easy: take the columns and their combinations |
| testing a given \(\mathbf v\) is easy: compute \(A\mathbf v\) | testing a given \(\mathbf v\) needs row operations on \([\,A\mid\mathbf v\,]\) |
| \(\operatorname{Nul} A=\{\mathbf 0\}\) iff \(A\mathbf x=\mathbf 0\) has only the trivial solution iff \(\mathbf x\mapsto A\mathbf x\) is one-to-one | \(\operatorname{Col} A=\mathbb R^m\) iff \(A\mathbf x=\mathbf b\) is consistent for every \(\mathbf b\) iff \(\mathbf x\mapsto A\mathbf x\) is onto \(\mathbb R^m\) |
Self-check only. Several answers are correct; the check tests whether \(\operatorname{Col} A=W\) for your matrix.
Let \(W=\{(a-2b,\ 3b,\ a+b,\ -a) : a,b \text{ real}\}\). Find a \(4\times 2\) matrix \(A\) with \(\operatorname{Col} A=W\).
2.5 Kernel and range of a linear transformation
A linear transformation \(T\) from a vector space \(V\) into a vector space \(W\) is a rule that assigns to each \(\mathbf x\) in \(V\) a unique vector \(T(\mathbf x)\) in \(W\), such that
The kernel of \(T\) is the set of all \(\mathbf u\) in \(V\) with \(T(\mathbf u)=\mathbf 0\). The range of \(T\) is the set of all vectors of the form \(T(\mathbf x)\) in \(W\). The kernel is a subspace of \(V\) and the range is a subspace of \(W\) (Figure 5).
If \(T(\mathbf x)=A\mathbf x\) for a matrix \(A\), then the kernel of \(T\) is \(\operatorname{Nul} A\) and the range of \(T\) is \(\operatorname{Col} A\).
Let \(\{p_k\}\) in \(\mathbb S\) be the daily price of a stock. The two-day moving average is the transformation \(M_2:\mathbb S\to\mathbb S\),
\[M_2(\{p_k\})=\left\{\frac{p_k+p_{k-1}}{2}\right\}.\]Show that \(M_2\) is linear and find its kernel.
Solution
Linear. For signals \(\{p_k\},\{q_k\}\) and a scalar \(c\):
\[M_2(\{p_k+q_k\})=\left\{\frac{p_k+p_{k-1}}{2}+\frac{q_k+q_{k-1}}{2}\right\}=M_2(\{p_k\})+M_2(\{q_k\}),\qquad M_2(\{cp_k\})=\left\{c\,\frac{p_k+p_{k-1}}{2}\right\}=c\,M_2(\{p_k\}).\]Kernel. \(\{p_k\}\) is in the kernel when \(p_k+p_{k-1}=0\), that is \(p_k=-p_{k-1}\), for every \(k\). Starting from \(p_0\) this gives \(p_k=p_0(-1)^k\). So the kernel is the set of multiples of the alternating signal \(\{(-1)^k\}=(\dots,1,-1,1,-1,\dots)\). The moving average removes exactly this day-to-day up-and-down pattern.
Day 3: Linearly Independent Sets; Bases
Today we find the smallest sets that still span a subspace, called bases, and we compute bases for \(\operatorname{Nul} A\), \(\operatorname{Col} A\) and \(\operatorname{Row} A\).
- Vectors \(\mathbf v_1,\dots,\mathbf v_p\) in \(\mathbb R^n\) are linearly independent exactly when \([\,\mathbf v_1\ \cdots\ \mathbf v_p\,]\mathbf x=\mathbf 0\) has only the trivial solution, that is, when every column of \([\,\mathbf v_1\ \cdots\ \mathbf v_p\,]\) is a pivot column.
- Two vectors are linearly dependent exactly when one is a multiple of the other. A set that contains \(\mathbf 0\) is linearly dependent. More than \(n\) vectors in \(\mathbb R^n\) are always linearly dependent.
- For an \(n\times n\) matrix \(A\), the following are equivalent: \(A\) is invertible; \(A\) has \(n\) pivots; \(\det A\neq 0\); the columns of \(A\) are linearly independent; the columns of \(A\) span \(\mathbb R^n\).
- \(\operatorname{Nul} A\) is the solution set of \(A\mathbf x=\mathbf 0\); \(\operatorname{Col} A\) is the span of the columns of \(A\); \(\operatorname{Row} A\) is the span of the rows of \(A\).
3.1 Linear independence in a vector space
An indexed set \(\{\mathbf v_1,\dots,\mathbf v_p\}\) in a vector space \(V\) is linearly independent if the vector equation
\[c_1\mathbf v_1+c_2\mathbf v_2+\cdots+c_p\mathbf v_p=\mathbf 0\]has only the trivial solution \(c_1=\cdots=c_p=0\). Otherwise the set is linearly dependent, and an equation with weights not all zero is a linear dependence relation.
When the vectors are matrices or functions, we cannot put them directly into the columns of a matrix. Then we work with the definition, or look for one vector that is a combination of the others.
In \(M_{2\times 2}\), let \(A_1=\begin{bmatrix} 1 & 0\\ 0 & 1\end{bmatrix}\), \(A_2=\begin{bmatrix} 0 & 1\\ 1 & 0\end{bmatrix}\), \(A_3=\begin{bmatrix} 1 & 1\\ 1 & 1\end{bmatrix}\). Then \(A_3=A_1+A_2\), so \(\{A_1,A_2,A_3\}\) is linearly dependent. The smaller set \(\{A_1,A_2\}\) is linearly independent: \(c_1A_1+c_2A_2=\begin{bmatrix} c_1 & c_2\\ c_2 & c_1\end{bmatrix}\) is the zero matrix only when \(c_1=c_2=0\).
3.2 Bases
Let \(H\) be a subspace of a vector space \(V\). A set \(\mathcal B\) of vectors in \(V\) is a basis for \(H\) if
In particular, every vector of \(\mathcal B\) must belong to \(H\).
Is \(\{\mathbf v_1,\mathbf v_2,\mathbf v_3\}\) a basis for \(\mathbb R^3\), where \(\mathbf v_1=(1,0,2)\), \(\mathbf v_2=(2,1,0)\), \(\mathbf v_3=(0,1,-3)\)?
Solution
There are three vectors in \(\mathbb R^3\), so the set is a basis exactly when \(A=[\,\mathbf v_1\ \mathbf v_2\ \mathbf v_3\,]\) is invertible. Use \(R_3\to R_3-2R_1\), then \(R_3\to R_3+4R_2\):
\[\begin{bmatrix} 1 & 2 & 0\\ 0 & 1 & 1\\ 2 & 0 & -3\end{bmatrix}\sim\begin{bmatrix} 1 & 2 & 0\\ 0 & 1 & 1\\ 0 & -4 & -3\end{bmatrix}\sim\begin{bmatrix} 1 & 2 & 0\\ 0 & 1 & 1\\ 0 & 0 & 1\end{bmatrix}.\]There are three pivots, so \(A\) is invertible and the set is a basis for \(\mathbb R^3\). (Equivalently, \(\det A=1\cdot 1\cdot 1=1\neq 0\).)
Enter two vectors \(\mathbf b_1,\mathbf b_2\) as the columns of the matrix and move the sliders. The figure shows \(c_1\mathbf b_1+c_2\mathbf b_2\), and the grid lines show the vectors with whole-number weights. When \(\mathbf b_1,\mathbf b_2\) are a basis, these grid lines cover the whole plane; when they are dependent, every combination stays on one line.
Self-check only.
For each set of vectors in \(\mathbb R^3\), choose the correct description.
Self-check only. Many answers are possible; the check tests whether your three vectors form a basis.
Find a vector \(\mathbf v_3\) such that \(\{(1,2,0),\ (0,1,-1),\ \mathbf v_3\}\) is a basis for \(\mathbb R^3\).
We need \(\det\begin{bmatrix} 1 & 0 & a\\ 2 & 1 & b\\ 0 & -1 & c\end{bmatrix}\neq 0\). Expanding down column 3: \(a(-2-0)-b(-1-0)+c(1-0)=-2a+b+c\). For \(\mathbf v_3=\mathbf e_3=(0,0,1)\) this is \(1\neq 0\), so \(\mathbf v_3=(0,0,1)\) works. Any \(\mathbf v_3\) with \(-2a+b+c\neq 0\) works; the vectors with \(-2a+b+c=0\) are exactly the combinations of the first two vectors.
3.3 The Spanning Set Theorem
A spanning set may contain vectors that are not needed. Removing them leaves the span unchanged.
Let \(\mathbf v_1=(1,0,1)\), \(\mathbf v_2=(0,1,1)\), \(\mathbf v_3=(2,1,3)\) and \(H=\operatorname{Span}\{\mathbf v_1,\mathbf v_2,\mathbf v_3\}\). Find a basis for \(H\).
Solution
\(2\mathbf v_1+\mathbf v_2=(2,0,2)+(0,1,1)=(2,1,3)=\mathbf v_3\). So \(\mathbf v_3\) is a combination of \(\mathbf v_1,\mathbf v_2\), and by Theorem 5(a) \(H=\operatorname{Span}\{\mathbf v_1,\mathbf v_2\}\). Every combination \(c_1\mathbf v_1+c_2\mathbf v_2+c_3\mathbf v_3\) can be rewritten as \((c_1+2c_3)\mathbf v_1+(c_2+c_3)\mathbf v_2\). The vectors \(\mathbf v_1,\mathbf v_2\) are not multiples of each other, so \(\{\mathbf v_1,\mathbf v_2\}\) is linearly independent and is a basis for \(H\). Geometrically, \(H\) is a plane and \(\mathbf v_3\) lies in it (Figure 7).
Find a basis for \(W=\operatorname{Span}\{\mathbf v_1,\mathbf v_2,\mathbf v_3,\mathbf v_4\}\), where
\[\mathbf v_1=\begin{bmatrix} 1\\ -3\\ 4\end{bmatrix},\quad \mathbf v_2=\begin{bmatrix} 6\\ 2\\ -1\end{bmatrix},\quad \mathbf v_3=\begin{bmatrix} 2\\ -2\\ 3\end{bmatrix},\quad \mathbf v_4=\begin{bmatrix} -4\\ -8\\ 9\end{bmatrix}.\]Solution
Put the vectors into a matrix and row reduce: \(R_2\to R_2+3R_1\), \(R_3\to R_3-4R_1\); then \(R_2\to\tfrac14R_2\), \(R_3\to-\tfrac15R_3\); then \(R_3\to R_3-R_2\).
\[\begin{bmatrix} 1 & 6 & 2 & -4\\ -3 & 2 & -2 & -8\\ 4 & -1 & 3 & 9\end{bmatrix}\sim \begin{bmatrix} 1 & 6 & 2 & -4\\ 0 & 20 & 4 & -20\\ 0 & -25 & -5 & 25\end{bmatrix}\sim \begin{bmatrix} 1 & 6 & 2 & -4\\ 0 & 5 & 1 & -5\\ 0 & 5 & 1 & -5\end{bmatrix}\sim \begin{bmatrix} 1 & 6 & 2 & -4\\ 0 & 5 & 1 & -5\\ 0 & 0 & 0 & 0\end{bmatrix}.\]The pivot columns are 1 and 2. So \(\mathbf v_3\) and \(\mathbf v_4\) are combinations of \(\mathbf v_1,\mathbf v_2\) (in fact \(\mathbf v_3=\tfrac45\mathbf v_1+\tfrac15\mathbf v_2\) and \(\mathbf v_4=2\mathbf v_1-\mathbf v_2\)), and \(\{\mathbf v_1,\mathbf v_2\}\) is a basis for \(W\).
3.4 Bases for \(\operatorname{Nul} A\), \(\operatorname{Col} A\) and \(\operatorname{Row} A\)
The method for writing \(\operatorname{Nul} A\) as a span (free variables as weights) always gives a linearly independent set. So when \(\operatorname{Nul} A\neq\{\mathbf 0\}\), it gives a basis for \(\operatorname{Nul} A\). For the other two subspaces we use the next two theorems.
The matrix \(A\) below is row equivalent to the reduced echelon form \(B\):
\[A=\begin{bmatrix} 1 & 4 & 0 & 2 & -1\\ 3 & 12 & 1 & 5 & 5\\ 2 & 8 & 1 & 3 & 2\\ 5 & 20 & 2 & 8 & 8\end{bmatrix}\sim B=\begin{bmatrix} 1 & 4 & 0 & 2 & 0\\ 0 & 0 & 1 & -1 & 0\\ 0 & 0 & 0 & 0 & 1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}.\]Find bases for \(\operatorname{Col} A\), \(\operatorname{Row} A\) and \(\operatorname{Nul} A\).
Solution
Col A. The pivots of \(B\) are in columns 1, 3, 5. In \(B\) we read off \(\mathbf b_2=4\mathbf b_1\) and \(\mathbf b_4=2\mathbf b_1-\mathbf b_3\). The columns of \(A\) satisfy the same relations: \(\mathbf a_2=4\mathbf a_1\) and \(\mathbf a_4=2\mathbf a_1-\mathbf a_3\) (check: \(2(1,3,2,5)-(0,1,1,2)=(2,5,3,8)=\mathbf a_4\)). The pivot columns of \(A\) form a basis:
\[\operatorname{Col} A:\ \left\{\begin{bmatrix} 1\\ 3\\ 2\\ 5\end{bmatrix},\ \begin{bmatrix} 0\\ 1\\ 1\\ 2\end{bmatrix},\ \begin{bmatrix} -1\\ 5\\ 2\\ 8\end{bmatrix}\right\}.\]Row A. By Theorem 7, the nonzero rows of \(B\) form a basis:
\[\operatorname{Row} A:\ \{(1,4,0,2,0),\ (0,0,1,-1,0),\ (0,0,0,0,1)\}.\]Nul A. From \(B\): \(x_2,x_4\) are free, \(x_1=-4x_2-2x_4\), \(x_3=x_4\), \(x_5=0\). So
\[\mathbf x=x_2\begin{bmatrix} -4\\ 1\\ 0\\ 0\\ 0\end{bmatrix}+x_4\begin{bmatrix} -2\\ 0\\ 1\\ 1\\ 0\end{bmatrix},\qquad \operatorname{Nul} A:\ \{(-4,1,0,0,0),\ (-2,0,1,1,0)\}.\]For \(\operatorname{Col} A\), use the pivot columns of \(A\) itself, not the columns of the echelon form. Row operations change the column space: in Example 7 every column of \(B\) has last entry 0, while \(\mathbf a_1=(1,3,2,5)\) does not, so the columns of \(B\) do not even lie in \(\operatorname{Col} A\).
Find a basis for the plane \(H=\{(x,y,z): x-3y+2z=0\}\) in \(\mathbb R^3\).
Solution
\(H=\operatorname{Nul}\begin{bmatrix} 1 & -3 & 2\end{bmatrix}\). The matrix is already in reduced echelon form; \(y,z\) are free and \(x=3y-2z\):
\[\begin{bmatrix} x\\ y\\ z\end{bmatrix}=y\begin{bmatrix} 3\\ 1\\ 0\end{bmatrix}+z\begin{bmatrix} -2\\ 0\\ 1\end{bmatrix}.\]A basis for \(H\) is \(\{(3,1,0),\ (-2,0,1)\}\). Check: \(3-3+0=0\) and \(-2-0+2=0\).
Self-check only. Several answers are correct; the check tests whether your vectors form a basis of each subspace.
Let \(A=\begin{bmatrix} 1 & 2 & 0 & 3\\ 2 & 4 & 1 & 4\\ -1 & -2 & 3 & -9\end{bmatrix}\). An echelon form of \(A\) is \(\begin{bmatrix} 1 & 2 & 0 & 3\\ 0 & 0 & 1 & -2\\ 0 & 0 & 0 & 0\end{bmatrix}\).
Self-check only. Several answers are correct.
3.5 Two views of a basis
Homework 9
- Fill in your name and surname below before you save the PDF.
- Show your work. For each step, name the property, the row operation or the reason you use. An answer without the steps that lead to it does not receive full credit.
- Every question has two answer modes: Type and Write by hand. In Write by hand mode the text boxes are replaced by a writing area where you can write with a pen, your finger or the mouse. For each question, only the answer of the selected mode goes into the PDF. Changing the mode does not delete what you entered in the other mode.
- In Type mode, write a vector as
(1, -2, 3)and a matrix as[1 2; 3 4](rows separated by semicolons). - To submit: press "Save homework as PDF", choose "Save as PDF" in the print dialog, and upload the file to Canvas.
- Your answers are saved automatically in this browser. If you open the page on a different computer or browser, they will not be there.
Question 115 points
Decide whether each set is a subspace of the given vector space. If it is, justify it (check the three properties, or write the set as a span or as a null space). If it is not, give a specific counterexample. Each part is worth 5 points.
Question 220 points
Let
\[A=\begin{bmatrix} 1 & -2 & 0 & 3 & -1\\ -2 & 4 & 1 & -8 & 6\\ 3 & -6 & 2 & 5 & 5\end{bmatrix}.\]R2 -> R2 + 2R1) and the new matrix. Then write the general solution as a combination of vectors with the free variables as weights. (9 points)
Question 320 points
The transformation \(T:M_{2\times 2}\to\mathbb R^2\) defined by \(T\left(\begin{bmatrix} a & b\\ c & d\end{bmatrix}\right)=\begin{bmatrix} a+d\\ b+c\end{bmatrix}\) is linear.
Question 420 points
Let
\[A=\begin{bmatrix} 1 & 3 & 0 & 0 & -2\\ 2 & 6 & 1 & 0 & -3\\ -1 & -3 & 1 & 1 & 6\\ 1 & 3 & 2 & -1 & -3\end{bmatrix}.\]