KAIST · Fall 2026MAS110 Practice

Chapter 4 · Geometry

Orthogonality and Approximation

One question

When b is out of reach, which output is closest?

Lecture 4

Four ideas behind the fit

Read the geometry first, then follow the same matrix through the code.

01Inner product
⟨u,v⟩=uTv\langle u,v\rangle = u^{\mathsf T}v

Use the standard Euclidean inner product to measure length and perpendicularity in this lab.

02Polynomial metric
⟨f,g⟩=∫−11f(t)g(t) dt=cTGd\langle f,g\rangle = \int_{-1}^{1} f(t)g(t)\,dt = c^{\mathsf T}Gd

Change the inner product from a dot product to an integral while keeping the same projection logic.

03Reduced QR
A=QR;QTQ=InA = QR;\quad Q^{\mathsf T}Q = I_n

Orthonormal columns give simple projection coordinates. For m>n,QQTm > n, QQ^{\mathsf T} is a projection, not ImI_{m}.

04Least squares
AT(b−Aθ^)=0A^{\mathsf T}(b-A\hat\theta)=0

Choose the nearest reachable output. Independent columns make the coefficients unique.

The running example · printed slide 67

A best fit still has a residual

For A=[[−1,1],[0,1],[1,0]]A = [[-1,1],[0,1],[1,0]] and b=(1,1,1)Tb = (1,1,1)^{\mathsf T}, no coefficient gives an exact fit. The nearest reachable output is b^=(2/3,4/3,2/3)T\hat{b} = (2/3,4/3,2/3)^{\mathsf T}.

b · observed
1
1
1
  1. 01θ^=(2/34/3)\hat\theta=\begin{pmatrix}2/3\\4/3\end{pmatrix}two coefficients
  2. 02b^=Aθ^\hat b=A\hat\thetathree fitted values
  3. 03AT(b−b^)=0A^{\mathsf T}(b-\hat b)=0a perpendicular residual
e=b−b^e=b-\hat b
1/3
−1/3
1/3

Chapter 4The fitted output lies in Col(A); the residual lies in its orthogonal complement.

Four guided labs

Build the geometry, then fit the data

  1. 4.1Inner Products, Angles, and ProjectionTurn a dot product into length, angle, and a perpendicular residual. See why QQTQQ^{\mathsf T} is a meaningful projection when Q is rectangular.dot product → norm → angle → projectionOpen lab
  2. 4.2Polynomial Inner Products and ApproximationTreat polynomials as vectors, compute the integral inner product on [−1,1], and remove the part of t2t^{2} already explained by lower-degree terms.coefficients → integral metric → orthonormal basisOpen lab
  3. 4.3Gram–Schmidt and Reduced QRSubtract projections one column at a time, reconstruct A=QRA = QR, and distinguish rectangular orthonormal columns from a square orthogonal matrix.subtract → normalize → reconstruct $A = QR$Open lab
  4. 4.4Least Squares as Orthogonal ProjectionFollow the lecture’s 3 × 2 example from an unreachable target to normal equations, a perpendicular residual, and a unique fitted output.minimize error → project b → check the residualOpen lab

One notation across the chapter

Keep the object and its coordinates separate

⟨⋅,⋅⟩ and ∥⋅∥\langle\cdot,\cdot\rangle\ \text{and}\ \lVert\cdot\rVert
The stated inner product determines the norm. In 4.1, 4.3, and 4.4 use the Euclidean dot product; in 4.2 use the integral on [−1,1]. The lecture’s |v| is written ∥v∥\Vert v\Vert here.
φ and θ^\varphi\ \text{and}\ \hat\theta
φ\varphi denotes an angle. θ\theta and θ^\hat{\theta} are candidate and minimizing model coefficients. Polynomial coordinate vectors use c,d instead of the function argument t.
A and GA\ \text{and}\ G
A stores Euclidean input columns. G is a Gram matrix of inner products: the polynomial metric in 4.2, and ATAA^{\mathsf T}A in 4.4. The lecture’s local polynomial Gram matrix A is renamed G here.
Q, R, PQ,\ R,\ P
Q stores orthonormal columns; R stores reconstruction coefficients; P acts on ambient vectors as a projection. Reduced Q is rectangular. QTQ=IQ^{\mathsf T}Q = I does not imply QQT=IQQ^{\mathsf T} = I.