Linear Algebra

where e _k has a 1 in the kth place and zeros elsewhere. All the figures above depicted points and vectors in R ² and R ³. Although it is not possible to draw such diagrams to illustrate geometric figures in R ⁿ if n > 3, it is possible to deal with them algebraically, and therein lies the real power of the algebraic machinery.

Example 1: Consider the vectors a = (1, 2, −3), b = (0, 1, −4, 2), and c = (5, −1, −1, 1) in R ⁴. Determine the vector 2 a − b + c.

Extend the definitions of scalar multiplication and vector addition in the natural way to vectors in R ⁴ to compute

Example 2: Determine the sum of the standard basis vectors e ₁, e ₃, and e ₄ in R ⁵.

All vectors in R ⁵ have five components. Four of the components in each of the standard basis vectors in R ⁵ are zero, and one component—the first in e ₁, the third in e ₃, and the fourth in e ₄—has the value 1. Therefore,

The norm of a vector. The length (or Euclidean norm) of a vector x is denoted ‖ x‖, and for a vector x = (x ₁, y₂) in R ², ‖ x‖ is easy to compute (see Figure ) by applying the Pythagorean Theorem: 

Figure 1

The expression for the length of a vector x = (x ₁, x ₂, x₃) in R ³ follows from two applications of the Pythagorean Theorem, as illustrated in Figure :

Figure 2

In general, the norm of a vector x = (x ₁, x ₂, x ₃,…, x _n) in R ⁿ is given by the equation 

Example 3: The length of the vector x = (3, 1, −5, 1) in R ⁴ is

Example 4: Let x be a vector in R ⁿ. If c is a scalar, how does the norm of c x compare to the norm of x?

If x = (x₁, x₂,…, x _n), then c x = (cx₁, cx₂,…, cx_n). Therefore,

Thus, multiplying a vector by a scalar c multiplies its norm by | c|. Note that this is consistent with the geometric description given earlier for scalar multiplication.

Distance between two points. The distance between two points x and y in R ⁿ —a quantity denoted by d(x, y)—is defined to be the length of the vector xy:

Example 5: What is the distance between the points p = (3, 1, 4) and q = (1, 3, 2)?

Since pq = q − p = (1, 3, 2) − (3, 1, 4) = (−2, 2, −2), the distance between the points p and q is 

Unit vectors. Any vector whose length is 1 is called a unit vector. Let x be a given nonzero vector and consider the scalar multiple x/‖ x‖. (The zero vector must be excluded from consideration here, for if x were 0, then ‖ x‖ would be 0, and the expression x/‖ x‖ would be undefined.) Applying the result of Example 4 (with c = 1/‖ x‖), the norm of the vector x/‖ x‖ is

Thus, for any nonzero vector x, is a unit vector. This vector is denoted ◯ (“x hat”) and represents the unit vector in the direction of x. (Indeed, one can go further and actually call ◯ the direction of x.) Note in particular that all the standard basis vectors are unit vectors; they are sometimes written as 

to emphasize this fact.

Example 6: Find the vector y in R ² whose length is 10 and which has the same direction as x = 3 i + 4 j.

The idea is simple: Find the unit vector in the same direction as 3 i + 4 j, and then multiply this unit vector by 10. The unit vector in the direction of x is

Therefore,

The dot product. One way to multiply two vectors—if they lie in R ³—is to form their cross product. Another way to form the product of two vectors—from the same space R ⁿ , for any n—is as follows. For any two n‐vectors x = (x₁, x₂,…, x_n) and y = (y₁, y₂,…, y_n), their dot product (or Euclidean inner product) is defined by the equation

(The symbol x·y is read “x dot y.”) Note carefully that, unlike the crooss product, the dot product of two vectors is a scalar. For this reason, the dot product is also called the scalar product. It can be easily shown that the dot product on R ⁿ satisfies the following useful identities: 

Example 7: What is the dot product of the vectors x=(−1, 0, 4) and y =(3, 6, 2) in R ³?

By the commutative property, it doesn't matter whether the product is taken to be x·y or y·x; the result is the same in either case. Applying the definition yields

The dot product of a vector x= (x ₁, x ₂,…, x _n) with itself is

Notice that the right‐hand side of this equation is also the expression for ‖ x‖ ²:

Therfore, for any vector x, 

This identity is put to use as follows. Since ‖ a‖ ² = a·a, the distributive and commutative properties of the dot product imply that for any vectors x and y in R ⁿ ,

Thus, ‖x + y ‖ ² = ‖x‖ ² + 2 x· y + ‖y‖ ² (*)

Now, if x⊥ y, then by figure , the Pythagorean Theorem would say

Figure 3

Therefore, if x⊥ y, equation (*) and (**) imply

which simplifies to the simple statement x·y = 0. Since this argument is reversible (assuming that it is agreed that the zero vector is orthogonal to every vector), the following fact has been established: 

This says that two vectors are orthogonal—that is, perpendicular—if and only if their dot product is zero.

Example 8: Use the dot product to verify that the cross product of the vectors x = (2,3,0) and y =(−1,1,4) is orthogonal to both x and y; then show that x x y is orthogonal to both x and y for any vectors x and y in R ³.

You can determine that x x y =(12,−8,5). The criterion for orthogonality is the vanishing of the dot product. Since both

and  

the vector x x y is indeed orthogonal to x and to y. In general,

and a similar calculation shows that (x x y)· x = 0 also.

The triangle inequality. From elementary geometry, you know that the sum of the lengths of any two sides of a triangle must be greater than the length of the third side. That is, if A, B, and C are the vertices of a triangle, then

This just says that the direct journey from A to C (along side AC) is shorter than the path from A to B and then to C; see Figure . This is called the triangle inequality.

Figure 4

The triangle inequality can be generalized to vectors in R ⁿ . If x and y are any two n‐vectors, then 

Figure shows that this statement can be interpreted in the same way as the elementary geometric fact about the lengths of the sides of a triangle. [One notable difference, however, is that if x and y happen to be parallel (that is, if y is a positive scalar times x) or if either x or y is the zero vector, then ‖ x + y‖ = ‖ x‖ + ‖ y‖. The generalized triangle inequality must take these degenerate cases into account (hence the weak inequality, ≤), whereas the triangle inequality from elementary geometry does not (and hence the strong (or strict) inequality, <).]

Figure 5

Example 9: Verify the triangle inequality for the vectors x= (−1, 0, 4) and y =(3, 6, 2) from Example 20.

The sum of these vectors is x + y = (2, 6, 6), and the lengths of the vectors x, y, and x + y are 

With these lengths, the triangle inequality, ‖ x + y‖≤‖ x‖+‖ y‖, becomes √76 ≤ √17+7, which is certainly true, since the lefthand side is less than 9, while the right‐hand side is greater than 4 + 7 = 11.

The Cauchy‐Schwarz inequality. One of the most important inequalities in mathematics is known as the Cauchy‐Schwarz inequality. For R ⁿ equipped with an inner product, this inequality states 

which says the absolute value of the dot product of two vectors is never greater than the product of their norms. Because of this inequality, it must be true that for any two nonzero vectors x and y,

Since both ‖ x‖ and ‖ y‖ are positive, the absolute value signs can be repositiones:

a statement which now directly implies

This final inequality says that there is precisely one value of θ between 0 and π (inclusive) such that

This θ is called the angle between the vectors x and y; geometrically, it is the smaller angle between them. (Note: No angle θ is defined if either x or y is the zero vector.) To verify that this θ is indeed the geometric angle between x and y, consider Figure , where it is assumed that the angle between x and y is acute.

Figure 6

The vector z is orthogonal to x, and the figure shows that y is the sum of z and a positive scalar multiple, c x, of x:

Taking the dot product of both sides of this equation with x yields

Since x and z are orthogonal, the dot product x·z is 0. This reduces the equation above to 

But Figure and the definition of cos θ indicate that

Now, since c is positive, ‖ c x‖ = | c| ‖ x‖ = c ‖ x‖, so this equation becomes

Equations (*) and (**), together with the identity x· x, then imply

which becomes

This proof can be extended to the case where the angle between x and y is obtuse, thus validating the following alternate—but entirely equivalent—definition of the dot product:

Note that this equation is consistent with the observation x ⊥ y ⟹ x· y = 0, since θ = π/2 implies cosθ = 0.

Example 10: Use the dot product to determine the angle between the vectors x = (2, 3, 0) and y = (−1, 1, 4).

From the boxed equation directly above,

Therefore, θ = cos ⁻¹ √1/234.

[Note that θ = sin ⁻¹ √233/234. Although this is consistent with the present calculation (because cos ² θ + sin ² θ must always equal 1 for any θ, and this is certainly true here), it is better to use the dot product than the cross product to determine the angle between two vectors in R ³. Why? The statement sinθ = √233/234 implies that θ is either 86.25 ^° or 93.75 ^°, and without further investigation, it is difficult to say which is correct. Even a picture may not help here; the angle is so close to 90 ^° that your drawing will probably not be accurate enough to tell the difference. But the statement cosθ = √1/234 says that θ is definitely 86.25 ^°, with no ambiguity. Within the range between 0 and 180 ^°, the sine function is entirely positive and cannot differentiate between an acute angle and its supplement. However, the cosine function is positive for acute angles and negative for obtuse angles, so it can—immediately—differentiate between an acute angle and its supplement.]

Orthogonal projections. Consider two nonzero vectors x and y emanating from the origin in R ⁿ . Dropping a perpendicular from the tip of x to the line containing y gives the (orthogonal) projection of x onto y. This vector is denoted proj _yx .

If θ < π/2 (Figure ), then the component of x along y, a positive scalar denoted comp _y x, is equal to the norm of the (vector) projection of x onto y. If θ > π/2 (Figure ), then the component of x along y is a negative scalar, equal to the negative of the norm of the projection of x onto y. (And if θ = π/2, then comp _y x = 0, since the orthogonal projection of x onto y is the zero vector.) In any case, the following equation holds:

where θ is the angle between x and y. Now, since x · y = ‖ x‖ ‖ y‖cosθ, this equation for the component of x along y can be rewritten as 

Figure 7

Figure 8

The vector projection of x onto y is equal to this scalar times the unit vector in the direction of y: 

Or, since ‖ y‖ ² = y·y,

Example 11: Find the projection of x = (2, 2, 4) onto the vector y = (2, 6, 3).

If θ is the angle between x and y, then the component of x along y is given by

Therefore,

See Figure .

Figure 9

Lines. A line is determined by two distinct points, say p and q. If the vector pq is drawn from p to q, then pq defines the line's direction. The description of a line can therefore be reformulated as follows: A line L is uniquely determined by

a given point through which L passes,