Lab 4.1

Inner Products, Angles, and Projection

Focus: dot product → norm → angle → projection
Notebook: Ch4-1 Inner Products in Euclidean Vector Spaces.ipynb

Lab goal

Compute lengths and angles, distinguish orthogonal from orthonormal, and derive a projection by making its residual perpendicular.

Lecture-note concepts

Real inner products
Cauchy–Schwarz
Euclidean norm
Orthogonal complements
Projection onto a line

Code extensionReplace unseeded random vectors with a reproducible 2D example; protect zero-vector normalization and interpret $QQ^{\mathsf T}$ correctly.

Lecture 4 · Reading guide

Read the symbols first

Wooseok Ha, MAS110 Lecture 4 (Fall 2026), lec04.pdf: printed slides 3–6, 12–25, 27, and 30 (PDF pages 5–8, 14–29, 31, and 34).

We keep all vectors real. An inner product is symmetric and bilinear, with $\langle u,u\rangle > 0$ for every nonzero u. Here the standard inner product is $u^{\mathsf T}v$ . Lab 4.2 keeps the same geometry but changes the inner product to an integral. Our deterministic example $v = (2,1)^{\mathsf T}$ and $u = (1,2)^{\mathsf T}$ makes every calculation inspectable.

$\langle u,v\rangle;\quad \lVert u\rVert;\quad u\perp v$: An inner product gives one real number, $\Vert u\Vert = \sqrt{\langle u,u\rangle}$ is its induced norm, and $u \perp v$ means $\langle u,v\rangle = 0$ . The lecture writes |u| for this norm. In Euclidean space the default is $u^{\mathsf T}v$ and the norm is $\Vert u\Vert _{2}$ ; for polynomials it is the stated integral norm.
$\varphi;\quad \theta;\quad \hat\theta$: This chapter uses $\varphi$ for an angle and $\theta$ for model coefficients in Lab 4.4. The lecture sometimes uses $\theta$ for the angle too; the distinction here prevents mixing an angle with a coefficient vector. A hat marks a minimizing coefficient, not a unit vector.
$A;\quad G;\quad Q;\quad R;\quad P$: A stores input columns in the Euclidean QR and least-squares labs. G denotes a Gram matrix of inner products, including the polynomial metric (which the lecture locally calls A). Q stores orthonormal columns, R stores their reconstruction coefficients, and P projects onto a subspace. These matrices have different jobs and shapes.
$u,v\in\mathbb R^2;\quad q=\frac{v}{\lVert v\rVert_2}$: u is the vector to project; v is a nonzero direction. q is its unit direction. Dividing by a norm requires a nonzero vector. The zero vector is orthogonal to every vector, but it has no direction or angle.
$\cos\varphi=\frac{u^{\mathsf T}v}{\lVert u\rVert_2\lVert v\rVert_2}$: The formula requires both vectors to be nonzero. Cauchy–Schwarz bounds the ratio by 1 in absolute value. Clip tiny floating-point excursions to [−1,1] before arccos; do not use clipping to hide a zero denominator.
$p=\lambda v;\quad \lambda=\frac{u^{\mathsf T}v}{v^{\mathsf T}v};\quad e=u-p$: $\lambda$ is a scalar coefficient, p is a vector on span{v}, and e is the perpendicular residual. For unit q the same vector is $p = (q^{\mathsf T}u)q$ . The coefficients $\lambda$ and $q^{\mathsf T}u$ differ because their basis vectors have different lengths.
$Q\in\mathbb R^{m\times k};\quad Q^{\mathsf T}Q=I_k$: Columns are orthonormal, so $k \le m$ . $QQ^{\mathsf T}$ is the m × m projection onto Col(Q), not a meaningless matrix. It equals $I_{m}$ only when the orthonormal columns span all of $\mathbb R^{m}$ ; an orthogonal matrix is square.
$W^\perp;\quad V=W\oplus W^\perp$: $W^{\perp}$ contains all vectors perpendicular to every vector of W. In finite-dimensional inner-product spaces each vector has a unique decomposition into these two subspaces. Two perpendicular subspaces need not span the whole space.
NumPy: @; np.outer; .T: For two 1D arrays @ is a dot product and returns a scalar. np.outer(q,q) makes the matrix $qq^{\mathsf T}$ . q.T alone still has shape (2,), so q @ q.T is a scalar, not the projection matrix.

Key ideas

Terms you will use

Inner product $\langle u,v\rangle = u^{\mathsf T}v$: A scalar comparison satisfying symmetry, bilinearity, and positive definiteness.
Keep in mind The inner product is a scalar; elementwise products form a vector until summed.
Norm and angle $\lVert u\rVert_2=\sqrt{u^{\mathsf T}u};\quad \cos\varphi=\frac{\langle u,v\rangle}{\lVert u\rVert_2\lVert v\rVert_2}$: A norm measures length; a normalized dot product compares directions.
Keep in mind Zero length cannot be normalized and has no angle.
Projection $p=(q^{\mathsf T}u)q;\quad e=u-p$: The component on a subspace leaving a perpendicular residual.
Keep in mind This short formula assumes q has unit norm.

Change the input

See the geometry respond

Keep $v = (2,1)^{\mathsf T}$ fixed. Rotate u relative to v or change its length. At 90° the projection vanishes; at length zero the angle is undefined.

Angle φ (degrees) 36.8699

\text{Length }\lVert u\rVert_2

2.23607

$\langle u,v\rangle$: 4
$\cos\varphi$: 0.8
$v^{\mathsf T}e$: 0

v · fixed direction
u · input
p · projection
e = u − p · residual

$u =$ (1, 2); $p =$ (1.6, 0.8); $e =$ (-0.6, 1.2). The perpendicularity check stays zero as the input changes.

Lab 4.1 · Euclidean geometry

From two vectors to a perpendicular residual

Use $u = (1,2)^{\mathsf T}$ and $v = (2,1)^{\mathsf T}$ to inspect each scalar, vector, and matrix. The formulas are the lecture’s; the small values are a reproducible teaching example.

Select a line to see its action, array shape, and concrete operation.

Maintained notebookCh4-1 Inner Products in Euclidean Vector Spaces.ipynb ↗The web trace shows prepared results. The geometry explorer recomputes them as inputs change.

L01 / 21Compute the dot product

line action

import numpy as np

Action: Loads NumPy.
Shape: module
Operation: np supplies arrays and linear algebra.

All traced lines

Compute the dot product
Turn squared lengths into lengths
Compute the angle safely
Project onto the line
Make the projection a matrix
Verify the orthogonal decomposition

Predict

What does u * v return, and what does u @ v return?

Before

Two vectors, one inner product

Both vectors have two entries. Their inner product will have just one.

u | v

After

Reveal to compare.

Connect the steps

Why the dot product determines the geometry

01 Cauchy–Schwarz makes the angle formula possible

If $v \ne 0, 0 \le \Vert u - tv\Vert _{2}^{2} = \Vert u\Vert _{2}^{2} - 2t\langle u,v\rangle + t^{2}\Vert v\Vert _{2}^{2}$ for every real t. At $t = \langle u,v\rangle /\Vert v\Vert _{2}^{2}$ , this implies $\langle u,v\rangle ^{2} \le \Vert u\Vert _{2}^{2}\Vert v\Vert _{2}^{2}$ . The $v = 0$ case gives $0 \le 0$ directly.

Thus the cosine ratio lies in [−1,1] for nonzero vectors. Scaling a vector by a positive number changes its length, but not its angle with another vector. A negative scale reverses its direction.

$|\langle u,v\rangle|\le\lVert u\rVert_2\lVert v\rVert_2;\quad \text{here }\cos\varphi=\frac45$

02 A unit direction changes the coordinate, not the line

The vectors v and $q = v/\Vert v\Vert _{2}$ span the same line. A coordinate along v is $\lambda = 4/5$ , while the coordinate along q is $q^{\mathsf T}u = 4/\sqrt{5}$ . Multiplying either coordinate by its associated basis vector gives $p = (8/5,4/5)^{\mathsf T}$ .

Orthogonal only means a zero inner product. Orthonormal additionally means unit length. Projection by an inner product alone uses a unit direction; otherwise the denominator $\langle v,v\rangle$ must remain.

$p=\frac{u^{\mathsf T}v}{v^{\mathsf T}v}\,v=(q^{\mathsf T}u)q$

03 The perpendicular residual certifies the nearest point

With $e = u - \lambda v$ and $\lambda = \langle u,v\rangle /\langle v,v\rangle , \langle e,v\rangle = 0$ . Any other point $\mu v$ differs from p only along the line, so its error is the sum of two perpendicular components.

Here $e = (-3/5,6/5)^{\mathsf T}$ , with squared length 9/5. Every $\mu$ other than 4/5 adds a positive squared distance along the line.

$\lVert u-\mu v\rVert_2^2=\lVert e\rVert_2^2+(\mu-\lambda)^2\lVert v\rVert_2^2$

04 $QQ^{\mathsf T}$ is a projection; completing Q changes its range

Because $Q^{\mathsf T}Q = I_{k}, P = QQ^{\mathsf T}$ satisfies $P^{\mathsf T} = P$ and $P^{2} = Q(Q^{\mathsf T}Q)Q^{\mathsf T} = P$ . Also Pz lies in Col(Q) and $Q^{\mathsf T}(z - Pz) = 0$ . That is exactly orthogonal projection.

The original 4.1 notebook calls a rectangular Q’s $QQ^{\mathsf T}$ meaningless. The corrected interpretation is a projection onto the chosen column space. Completing Q to a square orthogonal basis makes its range the whole space and $QQ^{\mathsf T} = I_{m}$ .

$q=\frac1{\sqrt5}\begin{pmatrix}2\\1\end{pmatrix}\ \Rightarrow\ P=\frac15\begin{pmatrix}4&2\\2&1\end{pmatrix}\ne I_2$

05 Perpendicular subspaces and complements are different claims

For $U = \operatorname{span}\{(1,0,0)^{\mathsf T}\}$ and $W = \operatorname{span}\{(0,1,0)^{\mathsf T}\}, U \perp W$ . But $U^{\perp}$ is the entire yz-plane; W is only one line in it. A complement must also account for the remaining direction.

For any finite-dimensional subspace W, projection gives $u = Pu + (I - P)u$ . The intersection W ∩ $W^{\perp}$ is {0}, since a vector perpendicular to itself has zero norm. This proves uniqueness of the decomposition.

Check your understanding

Is the zero vector perpendicular to v? Does it have an angle with v?

Its dot product with v is zero, so it is orthogonal. The angle is undefined because its norm is zero. The explorer reports this case explicitly.

Can you write the projection as

(u^{\mathsf T}v)v

Only if v is a unit vector. Otherwise divide by $v^{\mathsf T}v$ . For our v, this denominator is 5.

Are

\lambda

and p interchangeable?

No. $\lambda = 4/5$ is one coordinate; $p = \lambda v = (8/5,4/5)^{\mathsf T}$ is the projected vector. Changing the basis vector’s length changes the coordinate but not p.

Why is

QQ^{\mathsf T}

useful when Q has fewer columns than rows?

It projects onto Col(Q). $Q^{\mathsf T}Q$ checks orthonormality in coordinate space; $QQ^{\mathsf T}$ acts in the ambient space. They have different shapes and purposes.

Code references: NumPy: outer products ↗

Continue in Python

Run the complete notebook in Colab

This maintained notebook uses the same notation and worked example as the web lab. Run all cells in order, then change the inputs. NumPy is required; the final plot also uses Matplotlib, available in Colab.

Open in Colab ↗View the full source ↗