Linear Independence and Linear Transformations

Example 1

Let \(\mathbf v_1=\begin{bmatrix}1\\2\\3\end{bmatrix}\), \(\mathbf v_2=\begin{bmatrix}4\\5\\6\end{bmatrix}\), \(\mathbf v_3=\begin{bmatrix}2\\1\\0\end{bmatrix}\).

(a) Determine whether \(\{\mathbf v_1,\mathbf v_2,\mathbf v_3\}\) is linearly independent.
(b) If possible, find a linear dependence relation among \(\mathbf v_1,\mathbf v_2,\mathbf v_3\).

Solution.

(a) We must decide whether equation (1) has a nontrivial solution. Row reduce the augmented matrix. The operations are \(R_2\to R_2-2R_1\) and \(R_3\to R_3-3R_1\), then \(R_3\to R_3-2R_2\):

\[ \left[\begin{array}{ccc|c}1&4&2&0\\2&5&1&0\\3&6&0&0\end{array}\right]\sim \left[\begin{array}{ccc|c}1&4&2&0\\0&-3&-3&0\\0&-6&-6&0\end{array}\right]\sim \left[\begin{array}{ccc|c}1&4&2&0\\0&-3&-3&0\\0&0&0&0\end{array}\right] \]

Columns 1 and 2 are pivot columns, so \(x_1\) and \(x_2\) are basic variables and \(x_3\) is free. Every nonzero value of \(x_3\) gives a nontrivial solution of (1). Therefore \(\mathbf v_1,\mathbf v_2,\mathbf v_3\) are linearly dependent.

(b) To write down a dependence relation we need actual weights, so we continue to the reduced echelon form (\(R_2\to -\tfrac13R_2\), then \(R_1\to R_1-4R_2\)) and read off the system:

\[ \left[\begin{array}{ccc|c}1&0&-2&0\\0&1&1&0\\0&0&0&0\end{array}\right]\qquad \begin{aligned} x_1-2x_3&=0\\ x_2+x_3&=0\\ 0&=0\end{aligned} \]

So \(x_1=2x_3\), \(x_2=-x_3\), and \(x_3\) is free. Choose any nonzero value, say \(x_3=5\). Then \(x_1=10\), \(x_2=-5\), and

\[10\mathbf v_1-5\mathbf v_2+5\mathbf v_3=\mathbf 0.\]

Check: \(10\cdot1-5\cdot4+5\cdot2=0\), \(10\cdot2-5\cdot5+5\cdot1=0\), \(10\cdot3-5\cdot6+5\cdot0=0\). This is one of infinitely many possible dependence relations (any other nonzero choice of \(x_3\) gives another).

Example 2

Determine whether the columns of \(A=\begin{bmatrix}0&1&4\\1&2&-1\\5&8&0\end{bmatrix}\) are linearly independent.

Solution. We study \(A\mathbf x=\mathbf 0\). The right-hand side is zero and stays zero under row operations, so it is enough to row reduce \(A\) itself. First swap \(R_1\leftrightarrow R_2\) to get a nonzero entry in the top left, then \(R_3\to R_3-5R_1\), then \(R_3\to R_3+2R_2\):

\[ \begin{bmatrix}0&1&4\\1&2&-1\\5&8&0\end{bmatrix}\sim \begin{bmatrix}1&2&-1\\0&1&4\\5&8&0\end{bmatrix}\sim \begin{bmatrix}1&2&-1\\0&1&4\\0&-2&5\end{bmatrix}\sim \begin{bmatrix}1&2&-1\\0&1&4\\0&0&13\end{bmatrix} \]

There are three pivot columns and no free variables, so \(A\mathbf x=\mathbf 0\) has only the trivial solution. The columns of \(A\) are linearly independent.

Example 3

Determine whether the following sets are linearly independent.

(a) \(\mathbf v_1=\begin{bmatrix}3\\1\end{bmatrix}\), \(\mathbf v_2=\begin{bmatrix}6\\2\end{bmatrix}\)
(b) \(\mathbf v_1=\begin{bmatrix}3\\2\end{bmatrix}\), \(\mathbf v_2=\begin{bmatrix}6\\2\end{bmatrix}\)

Solution.

(a) Here \(\mathbf v_2=2\mathbf v_1\). Then \(-2\mathbf v_1+\mathbf v_2=\mathbf 0\) is a dependence relation with weights \(-2\) and \(1\), not both zero. So \(\{\mathbf v_1,\mathbf v_2\}\) is linearly dependent.

(b) Neither vector is a multiple of the other: a multiple \(c\mathbf v_1=\begin{bmatrix}3c\\2c\end{bmatrix}\) would need \(3c=6\) and \(2c=2\), that is \(c=2\) and \(c=1\) at the same time. Why does this settle the question? Suppose \(c\mathbf v_1+d\mathbf v_2=\mathbf 0\). If \(c\neq0\), we could solve for \(\mathbf v_1=(-d/c)\mathbf v_2\), making \(\mathbf v_1\) a multiple of \(\mathbf v_2\), which is false. So \(c=0\). Then \(d\mathbf v_2=\mathbf 0\) with \(\mathbf v_2\ne\mathbf 0\) forces \(d=0\). Only the trivial solution exists, so the set is linearly independent.

Example 4

Let \(\mathbf u=\begin{bmatrix}3\\2\\-4\end{bmatrix}\), \(\mathbf v=\begin{bmatrix}-6\\1\\7\end{bmatrix}\), \(\mathbf w=\begin{bmatrix}0\\-5\\2\end{bmatrix}\), \(\mathbf z=\begin{bmatrix}3\\7\\-5\end{bmatrix}\).

(a) Are the sets \(\{\mathbf u,\mathbf v\}\), \(\{\mathbf u,\mathbf w\}\), \(\{\mathbf u,\mathbf z\}\), \(\{\mathbf v,\mathbf w\}\), \(\{\mathbf v,\mathbf z\}\), \(\{\mathbf w,\mathbf z\}\) each linearly independent?
(b) Does (a) imply that \(\{\mathbf u,\mathbf v,\mathbf w,\mathbf z\}\) is linearly independent?
(c) Is \(\{\mathbf u,\mathbf v,\mathbf w,\mathbf z\}\) linearly dependent?
(d) Is \(\mathbf w\) a linear combination of \(\mathbf u,\mathbf v,\mathbf z\)?

Solution.

(a) Yes. In each pair, neither vector is a multiple of the other (compare the ratios of corresponding entries; for instance \(\mathbf u\) and \(\mathbf z\) agree in the first entry, so a multiple would need \(c=1\), but \(2\ne7\)). So each two-element set is linearly independent.

(b) No. Pairwise independence says nothing about the whole set. The four-element set could still be dependent.

(c) Yes. Four vectors in \(\mathbb R^3\) are always linearly dependent (this is Theorem 8 in Part 5 below; the reason is that the \(3\times4\) matrix \([\mathbf u\ \mathbf v\ \mathbf w\ \mathbf z]\) must have a free variable).

(d) No. First note that \(\mathbf z=3\mathbf u+\mathbf v\) (check: \(3\cdot3-6=3\), \(3\cdot2+1=7\), \(3\cdot(-4)+7=-5\)). So \(\operatorname{Span}\{\mathbf u,\mathbf v,\mathbf z\}=\operatorname{Span}\{\mathbf u,\mathbf v\}\), a plane. Is \(\mathbf w\) in that plane? We need \(a\mathbf u+b\mathbf v=\mathbf w\): the first entry gives \(3a-6b=0\), so \(a=2b\); the second gives \(2a+b=-5\), so \(5b=-5\), \(b=-1\), \(a=-2\). The third entry then gives \(-4(-2)+7(-1)=1\ne2\). No solution, so \(\mathbf w\) is not a combination of \(\mathbf u,\mathbf v,\mathbf z\), even though the set \(\{\mathbf u,\mathbf v,\mathbf w,\mathbf z\}\) is linearly dependent (\(\mathbf z\) is the vector that is a combination of the others). This is exactly the situation the warning describes: testing one chosen vector is not a safe way to test the whole set.

Example 5

The vectors \(\begin{bmatrix}2\\1\end{bmatrix}\), \(\begin{bmatrix}4\\-1\end{bmatrix}\), \(\begin{bmatrix}-2\\2\end{bmatrix}\) are linearly dependent by Theorem 8: three vectors, but only two entries each. Notice that none of the three is a multiple of another (Figure 2). The dependence relation exists anyway; it just involves all three vectors. For instance, solving \(x_1(2,1)+x_2(4,-1)=(-2,2)\) gives \(x_1=1\), \(x_2=-1\), so \(1\begin{bmatrix}2\\1\end{bmatrix}-1\begin{bmatrix}4\\-1\end{bmatrix}-1\begin{bmatrix}-2\\2\end{bmatrix}=\mathbf 0\).

Example 6

Determine by inspection whether each set is linearly dependent.

(a) \(\begin{bmatrix}1\\7\\6\end{bmatrix},\ \begin{bmatrix}2\\0\\9\end{bmatrix},\ \begin{bmatrix}3\\1\\5\end{bmatrix},\ \begin{bmatrix}4\\1\\8\end{bmatrix}\)
(b) \(\begin{bmatrix}2\\3\\5\end{bmatrix},\ \begin{bmatrix}0\\0\\0\end{bmatrix},\ \begin{bmatrix}1\\1\\8\end{bmatrix}\)
(c) \(\begin{bmatrix}-2\\4\\6\\10\end{bmatrix},\ \begin{bmatrix}3\\-6\\-9\\15\end{bmatrix}\)

Solution.

(a) Four vectors with three entries each: linearly dependent by Theorem 8.

(b) Theorem 8 does not apply (three vectors, three entries). But the set contains the zero vector, so it is linearly dependent by Theorem 9.

(c) Two vectors, so we check whether one is a multiple of the other. The second vector looks like \(-\tfrac32\) times the first: \(-\tfrac32(-2)=3\), \(-\tfrac32(4)=-6\), \(-\tfrac32(6)=-9\). But \(-\tfrac32(10)=-15\ne15\). The ratio fails in the fourth entry, so neither vector is a multiple of the other: the set is linearly independent.

Example 1

Let \(A=\begin{bmatrix}1&-3\\3&5\\-1&7\end{bmatrix}\), \(\mathbf u=\begin{bmatrix}2\\-1\end{bmatrix}\), \(\mathbf b=\begin{bmatrix}3\\2\\-5\end{bmatrix}\), \(\mathbf c=\begin{bmatrix}3\\2\\5\end{bmatrix}\), and define \(T:\mathbb R^2\to\mathbb R^3\) by \(T(\mathbf x)=A\mathbf x\), so that

\[T(\mathbf x)=\begin{bmatrix}1&-3\\3&5\\-1&7\end{bmatrix}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\begin{bmatrix}x_1-3x_2\\3x_1+5x_2\\-x_1+7x_2\end{bmatrix}.\]

(a) Find \(T(\mathbf u)\), the image of \(\mathbf u\).
(b) Find an \(\mathbf x\) in \(\mathbb R^2\) whose image under \(T\) is \(\mathbf b\).
(c) Is there more than one \(\mathbf x\) whose image is \(\mathbf b\)?
(d) Determine whether \(\mathbf c\) is in the range of \(T\).

Solution.

(a) Substitute \(x_1=2\), \(x_2=-1\):

\[T(\mathbf u)=A\mathbf u=\begin{bmatrix}2-3(-1)\\3(2)+5(-1)\\-2+7(-1)\end{bmatrix}=\begin{bmatrix}5\\1\\-9\end{bmatrix}.\]

(b) We need \(T(\mathbf x)=\mathbf b\), that is, \(A\mathbf x=\mathbf b\). Row reduce the augmented matrix: \(R_2\to R_2-3R_1\), \(R_3\to R_3+R_1\); then \(R_2\to\tfrac1{14}R_2\), \(R_3\to R_3-4R_2\); then \(R_1\to R_1+3R_2\):

\[ \left[\begin{array}{cc|c}1&-3&3\\3&5&2\\-1&7&-5\end{array}\right]\sim \left[\begin{array}{cc|c}1&-3&3\\0&14&-7\\0&4&-2\end{array}\right]\sim \left[\begin{array}{cc|c}1&-3&3\\0&1&-\tfrac12\\0&0&0\end{array}\right]\sim \left[\begin{array}{cc|c}1&0&\tfrac32\\0&1&-\tfrac12\\0&0&0\end{array}\right] \]

So \(x_1=\tfrac32\), \(x_2=-\tfrac12\), and \(\mathbf x=\begin{bmatrix}3/2\\-1/2\end{bmatrix}\). Check: \(\tfrac32+\tfrac32=3\), \(\tfrac92-\tfrac52=2\), \(-\tfrac32-\tfrac72=-5\).

(c) Every \(\mathbf x\) with image \(\mathbf b\) must solve the same system. The reduced matrix has a pivot in both columns, so there is no free variable and the solution is unique. Exactly one \(\mathbf x\) is sent to \(\mathbf b\).

(d) The vector \(\mathbf c\) is in the range of \(T\) if \(\mathbf c=T(\mathbf x)\) for some \(\mathbf x\), that is, if \(A\mathbf x=\mathbf c\) is consistent. Row reduce: \(R_2\to R_2-3R_1\), \(R_3\to R_3+R_1\); then \(R_3\to\tfrac14R_3\) and swap \(R_2\leftrightarrow R_3\); then \(R_3\to R_3-14R_2\):

\[ \left[\begin{array}{cc|c}1&-3&3\\3&5&2\\-1&7&5\end{array}\right]\sim \left[\begin{array}{cc|c}1&-3&3\\0&14&-7\\0&4&8\end{array}\right]\sim \left[\begin{array}{cc|c}1&-3&3\\0&1&2\\0&14&-7\end{array}\right]\sim \left[\begin{array}{cc|c}1&-3&3\\0&1&2\\0&0&-35\end{array}\right] \]

The last row says \(0=-35\), so the system is inconsistent. No \(\mathbf x\) is sent to \(\mathbf c\): \(\mathbf c\) is not in the range of \(T\).

Example 2: a shear

Let \(A=\begin{bmatrix}1&2\\0&1\end{bmatrix}\) and \(T(\mathbf x)=A\mathbf x\). Then \(T\begin{bmatrix}x_1\\x_2\end{bmatrix}=\begin{bmatrix}x_1+2x_2\\x_2\end{bmatrix}\): the second coordinate is unchanged, and the first coordinate is shifted by twice the second. Apply \(T\) to the corners of the square with vertices \((0,0),(2,0),(2,2),(0,2)\):

\[T\begin{bmatrix}0\\0\end{bmatrix}=\begin{bmatrix}0\\0\end{bmatrix},\quad T\begin{bmatrix}2\\0\end{bmatrix}=\begin{bmatrix}2\\0\end{bmatrix},\quad T\begin{bmatrix}2\\2\end{bmatrix}=\begin{bmatrix}6\\2\end{bmatrix},\quad T\begin{bmatrix}0\\2\end{bmatrix}=\begin{bmatrix}4\\2\end{bmatrix}.\]

Points on the base (\(x_2=0\)) stay where they are; points on the top edge (\(x_2=2\)) move \(4\) units to the right. Because \(T\) sends line segments to line segments (this follows from the linearity properties below), the whole square is mapped onto the parallelogram with these four vertices (Figure 4). Such a \(T\) is called a shear transformation.

Example 3: a rotation

Define \(T:\mathbb R^2\to\mathbb R^2\) by \(T(\mathbf x)=\begin{bmatrix}0&-1\\1&0\end{bmatrix}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\begin{bmatrix}-x_2\\x_1\end{bmatrix}\). Find the images of \(\mathbf u=\begin{bmatrix}4\\1\end{bmatrix}\), \(\mathbf v=\begin{bmatrix}2\\3\end{bmatrix}\) and \(\mathbf u+\mathbf v=\begin{bmatrix}6\\4\end{bmatrix}\).

Solution. Apply the rule \((x_1,x_2)\mapsto(-x_2,x_1)\) three times:

\[T(\mathbf u)=\begin{bmatrix}-1\\4\end{bmatrix},\qquad T(\mathbf v)=\begin{bmatrix}-3\\2\end{bmatrix},\qquad T(\mathbf u+\mathbf v)=\begin{bmatrix}-4\\6\end{bmatrix}.\]

Observe that \(T(\mathbf u+\mathbf v)=T(\mathbf u)+T(\mathbf v)\): \((-1,4)+(-3,2)=(-4,6)\). Figure 5 shows that \(T\) rotates each vector by \(90^\circ\) counterclockwise about the origin, and the parallelogram spanned by \(\mathbf u,\mathbf v\) goes to the parallelogram spanned by \(T(\mathbf u),T(\mathbf v)\).

Example 4: a dilation is linear

Define \(T:\mathbb R^2\to\mathbb R^2\) by \(T(\mathbf x)=3\mathbf x\). Show that \(T\) is a linear transformation.

Solution. We verify (4). Let \(\mathbf u,\mathbf v\) be in \(\mathbb R^2\) and let \(c,d\) be scalars. Then

\[ \begin{aligned} T(c\mathbf u+d\mathbf v)&=3(c\mathbf u+d\mathbf v)&&\text{definition of }T\\ &=3c\mathbf u+3d\mathbf v&&\text{vector arithmetic}\\ &=c(3\mathbf u)+d(3\mathbf v)&&\text{vector arithmetic}\\ &=cT(\mathbf u)+dT(\mathbf v)&&\text{definition of }T. \end{aligned} \]

So \(T\) satisfies (4) and is linear. A map \(\mathbf x\mapsto r\mathbf x\) is called a dilation when \(r>1\) and a contraction when \(0\le r\le1\); the same computation works for any \(r\).

Example 5: a transformation that is not linear

Column vectors are written as rows here. Define \(T:\mathbb R^2\to\mathbb R^3\) by \(T(x_1,x_2)=(2x_1-3x_2,\ x_1+4,\ 5x_2)\). Show that \(T\) is not linear.

Solution. A linear transformation must satisfy (3). But \(T(0,0)=(0,\,0+4,\,0)=(0,4,0)\ne(0,0,0)\). So \(T\) is not linear. (The "\(+4\)" in the second coordinate is the problem; without it, \(T\) would be the matrix transformation with matrix \(\begin{bmatrix}2&-3\\1&0\\0&5\end{bmatrix}\).)

Note that \(T(\mathbf 0)=\mathbf 0\) alone does not prove linearity; it is only a necessary condition. When (3) holds and you suspect \(T\) is not linear, look for a specific pair of vectors or a specific scalar for which (i) or (ii) fails.

Example 6: using linearity to compute images

Let \(T:\mathbb R^2\to\mathbb R^2\) be a linear transformation that maps \(\mathbf u=\begin{bmatrix}2\\1\end{bmatrix}\) into \(\begin{bmatrix}3\\4\end{bmatrix}\) and \(\mathbf v=\begin{bmatrix}1\\2\end{bmatrix}\) into \(\begin{bmatrix}1\\-5\end{bmatrix}\). Find the images of \(5\mathbf u\), \(4\mathbf v\) and \(5\mathbf u+4\mathbf v\).

Solution. We are not given a formula for \(T\), only two images. Linearity is enough. By (ii),

\[T(5\mathbf u)=5T(\mathbf u)=5\begin{bmatrix}3\\4\end{bmatrix}=\begin{bmatrix}15\\20\end{bmatrix},\qquad T(4\mathbf v)=4T(\mathbf v)=4\begin{bmatrix}1\\-5\end{bmatrix}=\begin{bmatrix}4\\-20\end{bmatrix}.\]

By (i),

\[T(5\mathbf u+4\mathbf v)=T(5\mathbf u)+T(4\mathbf v)=\begin{bmatrix}15\\20\end{bmatrix}+\begin{bmatrix}4\\-20\end{bmatrix}=\begin{bmatrix}19\\0\end{bmatrix}.\]

Example 7: from a formula to a matrix

Let \(\mathbf v_1=\begin{bmatrix}-3\\5\end{bmatrix}\), \(\mathbf v_2=\begin{bmatrix}2\\-9\end{bmatrix}\), and let \(T:\mathbb R^2\to\mathbb R^2\) map \(\mathbf x=\begin{bmatrix}x_1\\x_2\end{bmatrix}\) into \(x_1\mathbf v_1+x_2\mathbf v_2\). Find a matrix \(A\) such that \(T(\mathbf x)=A\mathbf x\) for every \(\mathbf x\).

Solution. The product \(A\mathbf x\) is \(x_1\) times the first column of \(A\) plus \(x_2\) times the second column. So we want a matrix whose columns are \(\mathbf v_1\) and \(\mathbf v_2\):

\[A=[\mathbf v_1\ \ \mathbf v_2]=\begin{bmatrix}-3&2\\5&-9\end{bmatrix},\qquad A\mathbf x=x_1\begin{bmatrix}-3\\5\end{bmatrix}+x_2\begin{bmatrix}2\\-9\end{bmatrix}=T(\mathbf x).\]

In particular \(T\) is a matrix transformation, so it is linear.