Introduction to Linear and Matrix Algebra

1. Vectors and Geometry

Photograph of a white directional arrow painted on grey asphalt pavement — Figure 1

A wide photographic banner at the top of the chapter opening page. It shows a close-up view of grey asphalt road surface with a large white painted arrow marking pointing up and to the left. The pavement texture is grainy and the upper left corner has a soft blue glow, suggesting daylight. Overlaid near the bottom center of the photograph is a light blue rectangular box with a dark border containing the chapter title text in bold dark lettering.

The power of mathematics is often to change one thing into another, to change geometry into language.

Marcus du Sautoy

This chapter serves as an introduction to the various objects—vectors, matrices, and linear transformations—that are the central focus of linear algebra. Instead of investigating what we can do with these objects, for now we simply focus on understanding their basic properties, how they interact with each other, and their geometric intuition.

1.1 Vectors and Vector Operations

In earlier math courses, focus was on how to manipulate expressions involving a single variable. For example, we learned how to solve equations like \( 4x - 3 = 7 \) and we learned about properties of functions like \( f(x) = 3x + 8 \), where in each case the one variable was called “\( x \)”. One way of looking at linear algebra is the natural extension of these ideas to the situation where we have two or more variables. For example, we might try solving an equation like \( 3x + 2y = 1 \), or we might want to investigate the properties of a function that takes in two independent variables and outputs two dependent variables.

The notation \( a \in S \) means that the object \( a \) is in the set \( S \), so \( \mathbf{v} \in \mathbb{R}^n \) means that the vector \( \mathbf{v} \) is in the set \( \mathbb{R}^n \) of \( n \)-dimensional space.

To make expressions involving several variables easier to deal with, we use vectors, which are ordered lists of numbers or variables. We say that the number of entries in the vector is its dimension, and if a vector has \( n \) entries, we say that it “lives in” or “is an element of” \( \mathbb{R}^n \). We denote vectors themselves by lowercase bold letters like \( \mathbf{v} \) and \( \mathbf{w} \), and we write their entries within parentheses. For example, \( \mathbf{v} = (2,3) \in \mathbb{R}^2 \) is a 2-dimensional vector and \( \mathbf{w} = (1,3,2) \in \mathbb{R}^3 \) is a 3-dimensional vector (just like \( 4 \in \mathbb{R} \) is a real number).

In the 2- and 3-dimensional cases, we can visualize vectors as arrows that indicate displacement in different directions by the amount specified in their entries. The vector’s first entry represents displacement in the \( x \)-direction, its second entry represents displacement in the \( y \)-direction, and in the 3-dimensional case its third entry represents displacement in the \( z \)-direction, as in Figure 1.1.

The front of a vector, where the tip of the arrow is located, is called its head, and the opposite end is called its tail. One way to compute the entries of a vector is to subtract the coordinates of its tail from the corresponding coordinates of its head. For example, the vector that goes from the point

N. Johnston, Introduction to Linear and Matrix Algebra,

https://doi.org/10.1007/978-3-030-52811-9_1

Two coordinate plots showing vector \( \mathbf{v} = (3,2) \) in the plane and \( \mathbf{v} = (1,3,2) \) in space — Figure 2

Two side-by-side diagrams. Panel (a), on the left, shows a two-dimensional Cartesian coordinate system with a horizontal \( x \)-axis labeled 0, 1, 2, 3 and a vertical \( y \)-axis labeled 0, 1, 2. A thick teal arrow starts at the origin and points up and to the right, ending at the point \( (3,2) \); dashed guide lines run from the arrowhead horizontally to the \( y \)-axis at 2 and vertically down to the \( x \)-axis at 3. The arrow is labeled \( \mathbf{v} = (3,2) \). The panel caption reads: (a) The vector \( \mathbf{v} = (3,2) \in \mathbb{R}^2 \). Panel (b), on the right, shows a three-dimensional coordinate system with a vertical \( z \)-axis marked 0, 1, 2, a \( y \)-axis receding to the right marked 1, 2, 3, and an \( x \)-axis coming forward toward the lower left marked 1. A thick teal arrow from the origin points up and to the right, ending at the point \( (1,3,2) \), with dashed lines forming a rectangular box that indicates the coordinates. The arrow is labeled \( \mathbf{v} = (1,3,2) \). The panel caption reads: (b) The vector \( \mathbf{v} = (1,3,2) \in \mathbb{R}^3 \).

Figure 1.1: Vectors can be visualized as arrows in (a) 2 and (b) 3 dimensions.

Some other books denote vectors with arrows like \( \vec{v} \), or \( \overrightarrow{AB} \) if they wish to specify that its tail is located at point A and its head is located at point B.

\( (-1,1) \) to the point \( (2,2) \) is \( (2,2) - (-1,1) = (3,1) \). However, this is also the same as the vector that points from \( (1,0) \) to \( (4,1) \), since \( (4,1) - (1,0) = (3,1) \) as well.

It is thus important to keep in mind that the coordinates of a vector specify its length and direction, but not its location in space; we can move vectors around in space without actually changing the vector itself, as in Figure 1.2. To remove this ambiguity when discussing vectors, we often choose to display them with their tail located at the origin—this is called the standard position of the vector.

When a vector is in standard position, the coordinates of the point at its head are exactly the same as the entries of the vector.

Plot showing three parallel arrows representing \( \mathbf{v} = (3,1) \) at different positions in the plane — Figure 3

A two-dimensional coordinate plot with a horizontal \( x \)-axis marked at -1, 0, 1, 2, 3, 4 and a vertical \( y \)-axis marked at 0, 1, 2. Three parallel arrows of identical length and slope point up and to the right, each rising 1 unit vertically for every 3 units horizontally. The topmost teal arrow starts near \( (-1,1) \) and ends near \( (2,2) \). A middle orange arrow starts at the origin \( (0,0) \) and ends at \( (3,1) \); it is labeled \( \mathbf{v} = (3,1) \) and is in standard position because its tail is at the origin. The lowest teal arrow starts near \( (1,0) \) and ends near \( (4,1) \). Faint horizontal gridlines run across at \( y = 0 \), \( y = 1 \), and \( y = 2 \).

Figure 1.2: Three copies of the vector \( \mathbf{v} = (3,1) \) located at different positions in the plane. The vector highlighted in orange is in standard position, since its tail is located at the origin.

1.1.1 Vector Addition

Even though we can represent vectors in 2 and 3 dimensions via arrows, we emphasize that one of our goals is to keep vectors (and all of our linear algebra tools) as dimension-independent as possible. Our visualizations involving arrows can thus help us build intuition for how vectors behave, but our definitions and theorems themselves should work just as well in \( \mathbb{R}^7 \) (even though we cannot really visualize this space) as they do in \( \mathbb{R}^3 \). For this reason, we typically introduce new concepts by first giving the algebraic, dimension-independent definition, followed by some examples to illustrate the geometric significance of the new concept. We start with vector addition, the simplest vector operation that there is.

Definition 1.1.1

Vector Addition

Suppose \( \mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n \) and \( \mathbf{w} = (w_1, w_2, \ldots, w_n) \in \mathbb{R}^n \) are vectors. Then their sum, denoted by \( \mathbf{v} + \mathbf{w} \), is the vector

\[ \mathbf{v} + \mathbf{w} \stackrel{\text{def}}{=} (v_1 + w_1, v_2 + w_2, \ldots, v_n + w_n). \]

Vector addition can be motivated in at least two different ways. On the one hand, it is algebraically the simplest operation that could reasonably be considered a way of adding up two vectors: most students, if asked to add up two vectors, would add them up entry-by-entry even if they had not seen Definition 1.1.1. On the other hand, vector addition also has a simple geometric picture in terms of arrows: If v and w are positioned so that the tail of w is located at the same point as the head of v (in which case we say that v and w are positioned head-to-tail), then \( \mathbf{v} + \mathbf{w} \) is the vector pointing from the tail of v to the head of w, as in Figure 1.3(a). In other words, \( \mathbf{v} + \mathbf{w} \) represents the total displacement accrued by following v and then following w.

If we instead work entirely with vectors in standard position, then \( \mathbf{v} + \mathbf{w} \) is the vector that points along the diagonal between sides v and w of a parallelogram, as in Figure 1.3(b).

Despite the triangle and parallelogram pictures looking different, the vector \( \mathbf{v} + \mathbf{w} \) is the same in each.

Two coordinate plane diagrams showing vector addition head-to-tail and in standard position — Figure 4

Two side-by-side diagrams on x-y coordinate axes illustrating vector addition. In diagram (a), labeled Adding vectors head-to-tail, a blue arrow labeled v points up and to the right from the origin, then a second blue arrow labeled w continues from the head of v further up and to the right; an orange arrow labeled v plus w runs from the origin directly to the head of w, forming the third side of a shaded triangle. In diagram (b), labeled Adding vectors in standard position, both blue arrows v and w start at the origin, v pointing up and to the right at a shallow angle and w pointing up and to the right at a steeper angle; dashed lines complete a shaded parallelogram, and an orange arrow labeled v plus w runs from the origin along the diagonal of the parallelogram to the opposite corner.

(a) Adding vectors head-to-tail. (b) Adding vectors in standard position.

Figure 1.3: How to visualize the addition of two vectors. If v and w are (a) positioned head-to-tail then \( \mathbf{v} + \mathbf{w} \) forms the third side of the triangle with sides v and w, but if v and w are (b) in standard position, then \( \mathbf{v} + \mathbf{w} \) is the diagonal of the parallelogram with sides v and w.

Before actually making use of vector addition, it will be useful to know some of the basic properties that it satisfies. We list two of the most important such properties in the following theorem for easy reference.

Theorem 1.1.1

Vector Addition Properties

Suppose \( \mathbf{v}, \mathbf{w}, \mathbf{x} \in \mathbb{R}^n \) are vectors. Then the following properties hold:

\( \mathbf{v} + \mathbf{w} = \mathbf{w} + \mathbf{v} \), and (commutativity)
\( (\mathbf{v} + \mathbf{w}) + \mathbf{x} = \mathbf{v} + (\mathbf{w} + \mathbf{x}) \). (associativity)

Proof. Both parts of this theorem can be proved directly by making use of the relevant definitions. To prove part (a), we use the definition of vector addition together with the fact that the addition of real numbers is commutative (i.e., \( x + y = y + x \) for all \( x, y \in \mathbb{R} \)):

\[ \begin{aligned} \mathbf{v} + \mathbf{w} &= (v_1 + w_1, v_2 + w_2, \ldots, v_n + w_n) \\ &= (w_1 + v_1, w_2 + v_2, \ldots, w_n + v_n) = \mathbf{w} + \mathbf{v}. \end{aligned} \]

The proof of part (b) of the theorem similarly follows fairly quickly from the definition of vector addition, and the corresponding property of real numbers, so we leave its proof to Exercise 1.1.14.

The two properties of vector addition that are described by Theorem 1.1.1 are called commutativity and associativity, respectively, and they basically say that we can unambiguously talk about the sum of any set of vectors without having to worry about the order in which we perform the addition. For example, this theorem shows that expressions like \( \mathbf{v} + \mathbf{w} + \mathbf{x} \) make sense, since there is no need to question whether it means \( (\mathbf{v} + \mathbf{w}) + \mathbf{x} \) or \( \mathbf{v} + (\mathbf{w} + \mathbf{x}) \).

While neither of these properties are surprising, it is still important to carefully think about which properties each vector operation satisfies as we introduce it. Later in this chapter, we will introduce two operations (matrix multiplication in Section 1.3.2 and the cross product in Section 1.A) that are not commutative (i.e., the order of "multiplication" matters since \( \mathbf{v} \times \mathbf{w} \ne \mathbf{w} \times \mathbf{v} \)), so it is important to be careful not to assume that basic properties like these hold without actually checking them first.

Example 1.1.1 Numerical Examples of Vector Addition

Compute the following vector sums:

\( (2,5,-1) + (1,-1,2) \),
\( (1,2) + (3,1) + (2,-1) \), and
the sum of the 8 vectors that point from the origin to the corners of a cube with opposite corners at \( (0,0,0) \) and \( (1,1,1) \), as shown:

Green cube in 3D axes with orange arrows from origin corner to each of the cube corners — Figure 5

A three-dimensional coordinate system with the z-axis pointing up, the y-axis pointing to the right, and the x-axis pointing toward the lower left. A shaded green cube is drawn with one corner at the origin and the opposite corner at the point \( (1,1,1) \). Seven orange arrows radiate from the origin corner outward to the other corners of the cube, representing the seven nonzero vectors from the origin to the cube corners. The eighth vector, \( (0,0,0) \), is not visible because it has zero length.

Even though we are adding 8 vectors, we can only see 7 vectors in the image. The missing vector that we cannot see is \( (0,0,0) \).

Solutions:

\( (2,5,-1) + (1,-1,2) = (2+1, 5-1, -1+2) = (3,4,1) \).
\( (1,2) + (3,1) + (2,-1) = (1+3+2, 2+1-1) = (6,2) \). Note that this sum can be visualized by placing all three vectors head-to-tail, as shown below. This same procedure works for any number of vectors.

Head-to-tail addition of vectors (1,2), (3,1), (2,-1) giving resultant (6,2) on xy-axes — Figure 6

A two-dimensional graph with a horizontal x-axis labeled from 0 to 6 and a vertical y-axis labeled 0 to 3. Three green arrows are placed head-to-tail: the first goes from the origin up and to the right to the point \( (1,2) \) and is labeled \( (1,2) \); the second goes from \( (1,2) \) to \( (4,3) \) and is labeled \( (3,1) \); the third goes from \( (4,3) \) down to \( (6,2) \) and is labeled \( (2,-1) \). A single long orange arrow drawn from the origin directly to the point \( (6,2) \) represents the resultant sum and is labeled \( (6,2) \).

Sums with lots of terms are often easier to evaluate if we can exploit some form of symmetry, as we do here in example (c).

We could list all 8 vectors and explicitly compute the sum, but a quicker method is to notice that the 8 vectors we are adding are exactly those that have any combination of 0's and 1's in their 3 entries (i.e., \( (0,0,1) \), \( (1,0,1) \), and so on). When we add them, in

any given entry, exactly half (i.e., 4) of the vectors have a 0 in that entry, and the other half have a 1 there. We thus conclude that the sum of these vectors is \( (4,4,4) \).

1.1.2 Scalar Multiplication

The other basic operation on vectors that we introduce at this point is one that changes a vector's length and/or reverses its direction, but does not otherwise change the direction in which it points.

Definition 1.1.2 Scalar Multiplication

“Scalar” just means “number”.

Suppose \( \mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n \) is a vector and \( c \in \mathbb{R} \) is a scalar. Then their scalar multiplication, denoted by \( c\mathbf{v} \), is the vector

\[ c\mathbf{v} \stackrel{\text{def}}{=} (cv_1, cv_2, \ldots, cv_n). \]

We remark that, once again, algebraically this is exactly the definition that someone would likely expect the quantity \( c\mathbf{v} \) to have. Multiplying each entry of \( \mathbf{v} \) by \( c \) seems like a rather natural operation, and it has the simple geometric interpretation of stretching \( \mathbf{v} \) by a factor of \( c \), as in Figure 1.4. In particular, if \( |c| > 1 \) then scalar multiplication stretches \( \mathbf{v} \), but if \( |c| < 1 \) then it shrinks \( \mathbf{v} \). When \( c < 0 \) then this operation also reverses the direction of \( \mathbf{v} \), in addition to any stretching or shrinking that it does if \( |c| \ne 1 \).

Graph with vector v = (2,1), stretched vector 2v, and reversed shrunken vector negative three-quarters v — Figure 7

A two-dimensional coordinate grid with a horizontal x-axis labeled with ticks at -2, -1, 0, 1, 2, 3, 4 and a vertical y-axis labeled with ticks at -1, 0, 1, 2. Three arrows all begin at the origin. An orange arrow labeled \( \mathbf{v} = (2,1) \) points up and to the right, ending at the point \( (2,1) \). A longer blue arrow labeled \( 2\mathbf{v} \) points in exactly the same direction but twice as far, ending near \( (4,2) \). A green arrow labeled \( \frac{-3}{4}\mathbf{v} \) points down and to the left, in the direction opposite to \( \mathbf{v} \) and shorter than it, ending near \( (-1.5,-0.75) \). The figure illustrates that multiplying a vector by a scalar stretches it, shrinks it, and/or reverses its direction.

Figure 1.4: Scalar multiplication can be used to stretch, shrink, and/or reverse the direction of a vector.

Two special cases of scalar multiplication are worth pointing out:

If \( c = 0 \) then \( c\mathbf{v} \) is the zero vector, all of whose entries are 0, which we denote by \( \mathbf{0} \).
If \( c = -1 \) then \( c\mathbf{v} \) is the vector whose entries are the negatives of \( \mathbf{v} \)'s entries, which we denote by \( -\mathbf{v} \).

In other words, vector subtraction is also performed in the “obvious” entrywise way.

We also define vector subtraction via \( \mathbf{v} - \mathbf{w} \stackrel{\text{def}}{=} \mathbf{v} + (-\mathbf{w}) \), and we note that it has the geometric interpretation that \( \mathbf{v} - \mathbf{w} \) is the vector pointing from the head of \( \mathbf{w} \) to the head of \( \mathbf{v} \) when \( \mathbf{v} \) and \( \mathbf{w} \) are in standard position. It is perhaps easiest to keep this geometric picture straight (“it points from the head of which vector to the head of the other one?”) if we just think of \( \mathbf{v} - \mathbf{w} \) as the vector that must be added to \( \mathbf{w} \) to get \( \mathbf{v} \) (so it points from \( \mathbf{w} \) to \( \mathbf{v} \)). Alternatively, \( \mathbf{v} - \mathbf{w} \) is the other diagonal (besides \( \mathbf{v} + \mathbf{w} \)) in the parallelogram with sides \( \mathbf{v} \) and \( \mathbf{w} \), as in Figure 1.5.

Parallelogram of vectors v and w with diagonals v plus w and v minus w on x y axes — Figure 8

A two-dimensional coordinate plane with a vertical axis labeled y and a horizontal axis labeled x, both drawn with arrowheads. From the origin, two blue arrows are drawn: one shorter arrow labeled w pointing up and to the right at a steep angle, and one arrow labeled v pointing to the right and slightly upward. A shaded light-blue parallelogram is formed with the origin as one corner and sides along v and w. The opposite corner of the parallelogram, at the upper right, is labeled v + w and is reached by dashed blue lines completing the parallelogram. An orange arrow labeled v - w runs from the head of w downward and to the right, ending at the head of v, illustrating that v - w is the other diagonal of the parallelogram. A faint light line also runs from the origin toward the v + w corner, showing that diagonal.

Figure 1.5: How to visualize the subtraction of two vectors. If \( \mathbf{v} \) and \( \mathbf{w} \) are in standard position then \( \mathbf{v} - \mathbf{w} \) is one of the diagonals of the parallelogram defined by \( \mathbf{v} \) and \( \mathbf{w} \) (and \( \mathbf{v} + \mathbf{w} \) is the other diagonal, as in Figure 1.3(b)).

It is straightforward to verify some simple properties of the zero vector, such as the facts that \( \mathbf{v} - \mathbf{v} = \mathbf{0} \) and \( \mathbf{v} + \mathbf{0} = \mathbf{v} \) for every vector \( \mathbf{v} \in \mathbb{R}^n \), by working entry-by-entry with the vector operations. There are also quite a few other simple ways in which scalar multiplication interacts with vector addition, some of which we now list explicitly for easy reference.

Theorem 1.1.2 Scalar Multiplication Properties

Suppose \( \mathbf{v}, \mathbf{w} \in \mathbb{R}^n \) are vectors and \( c, d \in \mathbb{R} \) are scalars. Then the following properties hold:

\( c(\mathbf{v} + \mathbf{w}) = c\mathbf{v} + c\mathbf{w} \),
\( (c + d)\mathbf{v} = c\mathbf{v} + d\mathbf{v} \), and
\( c(d\mathbf{v}) = (cd)\mathbf{v} \).

Property (a) says that scalar multiplication distributes over vector addition, and property (b) says that scalar multiplication distributes over real number addition.

Proof. All three parts of this theorem can be proved directly by making use of the relevant definitions. To prove part (a), we use the corresponding properties of real numbers in each entry of the vector:

\[ \begin{array}{rll} c(\mathbf{v} + \mathbf{w}) &= c(v_1 + w_1, v_2 + w_2, \ldots, v_n + w_n) & \text{(vector addition)} \\ &= \big(c(v_1 + w_1), c(v_2 + w_2), \ldots, c(v_n + w_n)\big) & \text{(scalar mult.)} \\ &= (cv_1 + cw_1, cv_2 + cw_2, \ldots, cv_n + cw_n) & \text{(property of } \mathbb{R}) \\ &= (cv_1, cv_2, \ldots, cv_n) + (cw_1, cw_2, \ldots, cw_n) & \text{(vector addition)} \\ &= c(v_1, v_2, \ldots, v_n) + c(w_1, w_2, \ldots, w_n) & \text{(scalar mult.)} \\ &= c\mathbf{v} + c\mathbf{w}. & \end{array} \]

The proofs of parts (b) and (c) of the theorem similarly follow fairly quickly from the definitions of vector addition and scalar multiplication, and the corresponding properties of real numbers, so we leave their proofs to Exercise 1.1.15. ■

Example 1.1.2 Numerical Examples of Vector Operations

Compute the indicated vectors:

\( 3\mathbf{v} - 2\mathbf{w} \), where \( \mathbf{v} = (2, 1, -1) \) and \( \mathbf{w} = (-1, 0, 3) \), and
the sum of the 6 vectors that point from the center \( (0,0) \) of a regular hexagon to its corners, one of which is located at \( (1,0) \), as shown:

Regular hexagon centered at origin with six vectors from center to corners, one at (1,0) — Figure 9

A coordinate plane with the vertical axis labeled y and the horizontal axis labeled x. A regular hexagon, shaded light green with a green outline, is centered at the origin. Six orange arrows are drawn from the center of the hexagon outward to each of its six corners. The corner on the positive x-axis, to the right of the origin, is labeled \( (1,0) \). The arrows point up-left, up-right, right, down-right, down-left, and left, so that the six vectors are evenly spaced around the center and occur in three opposite pairs.

This method of solving (b) has the nice feature that it still works even if we rotate the hexagon or change the number of sides.

Solutions:

\( 3\mathbf{v} - 2\mathbf{w} = (6,3,-3) - (-2,0,6) = (8,3,-9) \).
We could use trigonometry to find the entries of all 6 vectors explicitly, but an easier way to compute this sum is to label the vectors, in counter-clockwise order starting at an arbitrary location, as \( \mathbf{v}, \mathbf{w}, \mathbf{x}, -\mathbf{v}, -\mathbf{w}, -\mathbf{x} \) (since the final 3 vectors point in the opposite directions of the first 3 vectors). It follows that the sum is \( \mathbf{v} + \mathbf{w} + \mathbf{x} - \mathbf{v} - \mathbf{w} - \mathbf{x} = \mathbf{0} \).

By making use of these properties of vector addition and scalar multiplication, we can solve vector equations in much the same way that we solve equations involving real numbers: we can add and subtract vectors on both sides of an equation, and multiply and divide by scalars on both sides of the equation, until the unknown vector is isolated. We illustrate this procedure with some examples.

Example 1.1.3 Vector Algebra

Solve the following equations for the vector \( \mathbf{x} \):

\( \mathbf{x} - (3,2,1) = (1,2,3) - 3\mathbf{x} \), and
\( \mathbf{x} + 2(\mathbf{v} + \mathbf{w}) = -\mathbf{v} - 3(\mathbf{x} - \mathbf{w}) \).

Solutions:

We solve this equation as follows:
\[ \begin{array}{rrll} & \mathbf{x} - (3,2,1) &= (1,2,3) - 3\mathbf{x} & \\ \implies & \mathbf{x} &= (4,4,4) - 3\mathbf{x} & \text{(add } (3,2,1) \text{ to both sides)} \\ \implies & 4\mathbf{x} &= (4,4,4) & \text{(add } 3\mathbf{x} \text{ to both sides)} \\ \implies & \mathbf{x} &= (1,1,1). & \text{(divide both sides by 4)} \end{array} \]
The "\( \implies \)" symbol here is an implication arrow and is read as "implies". It means that the upcoming statement (e.g., \( \mathbf{x} = (1,1,1) \)) follows logically from the one before it (e.g., \( 4\mathbf{x} = (4,4,4) \)).
The method of solving this equation is the same as in part (a), but this time the best we can do is express \( \mathbf{x} \) in terms of \( \mathbf{v} \) and \( \mathbf{w} \):
\[ \begin{array}{rrll} & \mathbf{x} + 2(\mathbf{v} + \mathbf{w}) &= -\mathbf{v} - 3(\mathbf{x} - \mathbf{w}) & \\ \implies & \mathbf{x} + 2\mathbf{v} + 2\mathbf{w} &= -\mathbf{v} - 3\mathbf{x} + 3\mathbf{w} & \text{(expand parentheses)} \\ \implies & 4\mathbf{x} &= -3\mathbf{v} + \mathbf{w} & \text{(add } 3\mathbf{x}, \text{ subtract } 2\mathbf{v} + 2\mathbf{w}) \\ \implies & \mathbf{x} &= \tfrac{1}{4}(\mathbf{w} - 3\mathbf{v}). & \text{(divide both sides by 4)} \end{array} \]

1.1.3 Linear Combinations

One common task in linear algebra is to start out with some given collection of vectors \( \mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k \) and then use vector addition and scalar multiplication to construct new vectors out of them. The following definition gives a name to this concept.

Definition 1.1.3 Linear Combinations

A linear combination of the vectors \( \mathbf{v}_1, \mathbf{v}_2, \ldots, \mathbf{v}_k \in \mathbb{R}^n \) is any vector of the form

\[ c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k, \]

where \( c_1, c_2, \ldots, c_k \in \mathbb{R} \).

We will see how to determine whether or not a vector is a linear combination of a given set of vectors in Section 2.1.

For example, \( (1,2,3) \) is a linear combination of the vectors \( (1,1,1) \) and \( (-1,0,1) \) since \( (1,2,3) = 2(1,1,1) + (-1,0,1) \). On the other hand, \( (1,2,3) \) is not a linear combination of the vectors \( (1,1,0) \) and \( (2,1,0) \) since every vector of the form \( c_1(1,1,0) + c_2(2,1,0) \) has a 0 in its third entry, and thus cannot possibly equal \( (1,2,3) \).

When working with linear combinations, some particularly important vectors are those with all entries equal to 0, except for a single entry that equals 1. Specifically, for each \( j = 1, 2, \ldots, n \), we define the vector \( \mathbf{e}_j \in \mathbb{R}^n \) by

\[ \mathbf{e}_j \stackrel{\text{def}}{=} (0, 0, \ldots, 0, \underset{\uparrow\, j\text{-th entry}}{1}, 0, \ldots, 0). \]

Whenever we use these vectors, the dimension of \( \mathbf{e}_j \) will be clear from context or by saying things like \( \mathbf{e}_3 \in \mathbb{R}^7 \).

For example, in \( \mathbb{R}^2 \) there are two such vectors: \( \mathbf{e}_1 = (1,0) \) and \( \mathbf{e}_2 = (0,1) \). Similarly, in \( \mathbb{R}^3 \) there are three such vectors: \( \mathbf{e}_1 = (1,0,0) \), \( \mathbf{e}_2 = (0,1,0) \), and \( \mathbf{e}_3 = (0,0,1) \). In general, in \( \mathbb{R}^n \) there are \( n \) of these vectors, \( \mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_n \), and we call them the standard basis vectors (for reasons that we discuss in the next chapter). Notice that in \( \mathbb{R}^2 \) and \( \mathbb{R}^3 \), these are the vectors that point a distance of 1 in the direction of the x-, y-, and z-axes, as in Figure 1.6.

Two coordinate diagrams showing standard basis vectors in the plane and in space. — Figure 10

Two side-by-side coordinate diagrams. Panel (a), labeled "The standard basis vectors \( \mathbf{e}_1 \) and \( \mathbf{e}_2 \) in \( \mathbb{R}^2 \)", shows a two-dimensional coordinate system with a horizontal x-axis marked 0, 1, 2 and a vertical y-axis marked 0, 1, 2. Two orange arrows start at the origin: one points right one unit along the x-axis and is labeled \( \mathbf{e}_1 = (1,0) \); the other points up one unit along the y-axis and is labeled \( \mathbf{e}_2 = (0,1) \). Panel (b), labeled "The standard basis vectors \( \mathbf{e}_1, \mathbf{e}_2 \), and \( \mathbf{e}_3 \) in \( \mathbb{R}^3 \)", shows a three-dimensional coordinate system with the z-axis pointing up (marked 1 and 2), the y-axis going to the right (marked 1 and 2), and the x-axis coming toward the lower left (marked 1 and 2). Three orange arrows emanate from the origin, one along each axis: \( \mathbf{e}_3 = (0,0,1) \) upward along z, \( \mathbf{e}_2 = (0,1,0) \) to the right along y, and \( \mathbf{e}_1 = (1,0,0) \) toward the viewer along x. Each arrow has length one unit.

(a) The standard basis vectors \( \mathbf{e}_1 \) and \( \mathbf{e}_2 \) in \( \mathbb{R}^2 \).

(b) The standard basis vectors \( \mathbf{e}_1, \mathbf{e}_2 \), and \( \mathbf{e}_3 \) in \( \mathbb{R}^3 \).

Figure 1.6: The standard basis vectors point a distance of 1 along the x-, y-, and z-axes.

For now, the reason for our interest in these standard basis vectors is that every vector \( \mathbf{v} \in \mathbb{R}^n \) can be written as a linear combination of them. In particular, if \( \mathbf{v} = (v_1, v_2, \ldots, v_n) \) then

\[ \mathbf{v} = v_1\mathbf{e}_1 + v_2\mathbf{e}_2 + \cdots + v_n\mathbf{e}_n, \]

When we see expressions like this, it is useful to remind ourselves of the "type" of each object: \( v_1, v_2, \ldots, v_n \) are scalars and \( \mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_n \) are vectors.

which can be verified just by computing each of the entries of the linear combination on the right. This idea of writing vectors in terms of the standard basis vectors (or other distinguished sets of vectors that we introduce later) is one of the most useful techniques that we make use of in linear algebra: in many situations, if we can prove that some property holds for the standard basis vectors, then we can use linear combinations to show that it must hold for all vectors.

Exercises

Example 1.1.4 Numerical Examples of Linear Combinations

Compute the indicated linear combinations of standard basis vectors:

Compute \( 3\mathbf{e}_1 - 2\mathbf{e}_2 + \mathbf{e}_3 \in \mathbb{R}^3 \), and
Write \( (3,5,-2,-1) \) as a linear combination of \( \mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3, \mathbf{e}_4 \in \mathbb{R}^4 \).

Solutions:

\( 3\mathbf{e}_1 - 2\mathbf{e}_2 + \mathbf{e}_3 = 3(1,0,0) - 2(0,1,0) + (0,0,1) = (3,-2,1) \). In general, when adding multiples of the standard basis vectors, the resulting vector has the coefficient of \( \mathbf{e}_1 \) in its first entry, the coefficient of \( \mathbf{e}_2 \) in its second entry, and so on.
Just like in part (a), the entries of the vectors are the scalars in the linear combination: \( (3,5,-2,-1) = 3\mathbf{e}_1 + 5\mathbf{e}_2 - 2\mathbf{e}_3 - \mathbf{e}_4 \).

Remark 1.1.1 No Vector Multiplication

At this point, it seems natural to ask why we have defined vector addition \( \mathbf{v} + \mathbf{w} \) and scalar multiplication \( c\mathbf{v} \) in the "obvious" entrywise ways, but we have not similarly defined the entrywise product of two vectors:

\[ \mathbf{v}\mathbf{w} \stackrel{\text{def}}{=} \left( v_1 w_1, v_2 w_2, \ldots, v_n w_n \right). \]

The answer is simply that entrywise vector multiplication is not particularly useful—it does not often come up in real-world problems or play a role in more advanced mathematical structures, nor does it have a simple geometric interpretation. There are some other more useful ways of "multiplying" vectors together, called the dot product and the cross product, which we explore in Sections 1.2 and 1.A, respectively.

solutions to starred exercises on page 435

1.1.1 Draw each of the following vectors in standard position in \( \mathbb{R}^2 \):

∗(a) \( \mathbf{v} = (3,2) \)
(b) \( \mathbf{w} = (-0.5, 3) \)
∗(c) \( \mathbf{x} = (1,-3) \)
(d) \( \mathbf{y} = (-2,-1) \)

∗1.1.2 Draw each of the vectors from Exercise 1.1.1, but with their tail located at the point \( (1,2) \).

∗1.1.3 If each of the vectors from Exercise 1.1.1 are positioned so that their heads are located at the point \( (3,3) \), find the location of their tails.

1.1.4 Draw each of the following vectors in standard position in \( \mathbb{R}^3 \):

∗(a) \( \mathbf{v} = (0,0,2) \)
(b) \( \mathbf{w} = (-1,2,1) \)
∗(c) \( \mathbf{x} = (1,2,0) \)
(d) \( \mathbf{y} = (3,2,-1) \)

1.1.5 If the vectors \( \mathbf{v}, \mathbf{w}, \mathbf{x} \), and \( \mathbf{y} \) are as in Exercise 1.1.1, then compute

∗(a) \( \mathbf{v} + \mathbf{w} \)
(b) \( \mathbf{v} + \mathbf{w} + \mathbf{y} \)
∗(c) \( \mathbf{y} - 2\mathbf{x} \)
(d) \( \mathbf{v} + 2\mathbf{w} + 2\mathbf{x} + 2\mathbf{y} \)

1.1.6 If the vectors \( \mathbf{v}, \mathbf{w}, \mathbf{x} \), and \( \mathbf{y} \) are as in Exercise 1.1.4, then compute

∗(a) \( \mathbf{v} + \mathbf{y} \)
(b) \( 4\mathbf{w} + 3\mathbf{w} - (2\mathbf{w} + 6\mathbf{w}) \)
∗(c) \( 4\mathbf{x} - 2\mathbf{w} \)
(d) \( 2\mathbf{x} - \mathbf{w} - \mathbf{y} \)

∗1.1.7 Write each of the vectors \( \mathbf{v}, \mathbf{w}, \mathbf{x} \), and \( \mathbf{y} \) from Exercise 1.1.4 as a linear combination of the standard basis vectors \( \mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3 \in \mathbb{R}^3 \).

1.1.8 Suppose that the side vectors of a parallelogram are \( \mathbf{v} = (1,4) \) and \( \mathbf{w} = (-2,1) \). Find vectors describing both of the parallelogram's diagonals.

∗1.1.9 Suppose that the diagonal vectors of a parallelogram are \( \mathbf{x} = (3,-2) \) and \( \mathbf{y} = (1,4) \). Find vectors describing the parallelogram's sides.

1.1.10 Solve the following vector equations for \( \mathbf{x} \):

∗(a) \( (1,2) - \mathbf{x} = (3,4) - 2\mathbf{x} \)
(b) \( 3((1,-1) + \mathbf{x}) = 2\mathbf{x} \)
∗(c) \( 2(\mathbf{x} + 2(\mathbf{x} + 2\mathbf{x})) = 3(\mathbf{x} + 3(\mathbf{x} + 3\mathbf{x})) \)
(d) \( -2(\mathbf{x} - (1,-2)) = \mathbf{x} + 2(\mathbf{x} + (1,1)) \)

1.1.11 Write the vector \( \mathbf{x} \) in terms of the vectors \( \mathbf{v} \) and \( \mathbf{w} \):

∗(a) \( \mathbf{v} - \mathbf{x} = \mathbf{w} + \mathbf{x} \)
(b) \( 2\mathbf{v} - 3\mathbf{x} = 4\mathbf{x} - 5\mathbf{w} \)
∗(c) \( 4(\mathbf{x} + \mathbf{v}) - \mathbf{x} = 2(\mathbf{w} + \mathbf{x}) \)
(d) \( 2(\mathbf{x} + 2(\mathbf{x} + 2\mathbf{x})) = 2(\mathbf{v} + 2\mathbf{v}) \)

∗1.1.12 Does there exist a scalar \( c \in \mathbb{R} \) such that \( c(1,2) = (3,4) \)? Justify your answer both algebraically and geometrically.

1.1.13 Let \( n \ge 3 \) be an integer and consider the set of \( n \) vectors that point from the center of the regular \( n \)-gon in \( \mathbb{R}^2 \) to its corners.

(a) Show that if \( n \) is even then the sum of these \( n \) vectors is 0. [Hint: We solved the \( n = 6 \) case in Example 1.1.2(b).]
(b) Show that if \( n \) is odd then the sum of these \( n \) vectors is 0. [Hint: This is more difficult. Try working with the \( x \)- and \( y \)-entries of the sum individually.]

∗∗1.1.14 Prove part (b) of Theorem 1.1.1.

∗∗1.1.15 Recall Theorem 1.1.2, which established some of the basic properties of scalar multiplication.

(a) Prove part (b) of the theorem.
(b) Prove part (c) of the theorem.

1.2 Lengths, Angles, and the Dot Product

When discussing geometric properties of vectors, like their length or the angle between them, we would like our definitions to be as dimension-independent as possible, so that it is just as easy to discuss the length of a vector in \( \mathbb{R}^7 \) as it is to discuss the length of one in \( \mathbb{R}^2 \). At first it might be somewhat surprising that discussing the length of a vector in high-dimensional spaces is something that we can do at all—after all, we cannot really visualize anything past 3 dimensions. We thus stress that the dimension-independent definitions of length and angle that we introduce in this section are not theorems that we prove, but rather are definitions that we adopt so that they satisfy the basic geometric properties that lengths and angles “should” satisfy.

1.2.1 The Dot Product

The main tool that helps us extend geometric notions from \( \mathbb{R}^2 \) and \( \mathbb{R}^3 \) to arbitrary dimensions is the dot product, which is a way of combining two vectors so as to create a single number:

Definition 1.2.1 Dot Product

Suppose \( \mathbf{v} = (v_1, v_2, \ldots, v_n) \in \mathbb{R}^n \) and \( \mathbf{w} = (w_1, w_2, \ldots, w_n) \in \mathbb{R}^n \) are vectors. Then their dot product, denoted by \( \mathbf{v} \cdot \mathbf{w} \), is the quantity

\[ \mathbf{v} \cdot \mathbf{w} \stackrel{\text{def}}{=} v_1 w_1 + v_2 w_2 + \cdots + v_n w_n. \]

It is important to keep in mind that the output of the dot product is a number, not a vector. So, for example, the expression \( \mathbf{v} \cdot (\mathbf{w} \cdot \mathbf{x}) \) does not make sense, since \( \mathbf{w} \cdot \mathbf{x} \) is a number, and so we cannot take its dot product with \( \mathbf{v} \). On the other hand, the expression \( \mathbf{v} / (\mathbf{w} \cdot \mathbf{x}) \) does make sense, since dividing a vector by a number is a valid mathematical operation. As we introduce more operations between different types of objects, it will become increasingly important to keep in mind the type of object that we are working with at all times.

Example 1.2.1 Numerical Examples of the Dot Product

Compute (or state why it's impossible to compute) the following dot products:

a) \( (1,2,3) \cdot (4,-3,2) \),
b) \( (3,6,2) \cdot (-1,5,2,1) \), and