Mathematical Foundations

This chapter explains the ideas and numerical principles that make GAMs work, exploring not just the equations, but why we chose each ingredient and what problem it solves.

magli provides two distinct engines that solve the same penalized regression problem with different statistical machinery. The mathematical foundations are organized into four self-contained chapters:

• Part I — Shared Foundations
  • Why GAMs?
  • Decomposition — How the Model Explains Its Predictions
  • Core Intuitions and Mental Models
  • A Practical Guide to Model Diagnostics
• Part II — The Eilers & Marx Backend
  • How the Eilers-Marx backend thinks
  • B-Spline Bases and Local Building Blocks
  • Penalties and the Cost of Flexibility
  • The SVD data-augmentation solver
  • Effective Degrees of Freedom (EDF)
  • Generalised Cross Validation (GCV)
  • Putting the Penalized Fit Together
  • Tensor-product interactions
  • Confidence and prediction intervals
  • How Fast Grid Search Works
  • Observation weights
• Part III — The Wood Backend
  • How the Wood backend thinks
  • The basis zoo
  • Identifiability, Rescaling, and the Sum-to-Zero Constraint
  • Smoothing Parameter Selection with REML and GCV.Cp
  • Inference with Two Covariances and Wood’s F-Test
• Part IV — Backends Compared
  • Two Philosophies for the Same Goal
  • The Choice of Basis — Uniform LEGO Bricks vs. Tailored Geometries
  • Tuning Smoothness — Discrete Grid Sweeps vs. Continuous Mountain Climbing
  • Numerical Solvers — SVD Data Augmentation vs. Cholesky Efficiency
  • Identifiability and Zero-Centering — Implicit Offsets vs. Absorbed Constraints
  • Statistical Inference and Hypothesis Testing
  • Physical Shape Constraints — When Domain Knowledge Trumps Noise
  • How to Choose the Right Backend for Your Problem

Overview & Chapter Guide

• Part I — Shared Foundations (Part I) introduces the shared ideas both backends agree on, explaining what a GAM is, why additive models balance flexibility with interpretability, and how predictions decompose into independent component plots.

• Part II — The Eilers & Marx Backend (Part II) covers the Eilers-Marx backend, detailing uniform B-splines, discrete difference penalties, SVD data-augmented solver, Generalized Cross Validation (GCV), fast QR+Cholesky grid search, and observation weights.

• Part III — The Wood Backend (Part III) covers the Wood backend, detailing Simon Wood’s rich basis families (thin plate, cubic, cyclic, P-splines, random effects), null-space constraint absorption, sum-to-zero identifiability, and continuous REML/GCV optimization with approximate F-tests and p-values (Wood 2013).

• Part IV — Backends Compared (Part IV) provides an in-depth side-by-side comparison of the two backends (numerical methods, scaling, and decision principles) so you can choose the optimal engine for your data.

Where to Start?

• If you’re new to GAMs, start with Part I — Shared Foundations.

• If you already understand penalized splines and want implementation details, skip to the SVD solver (Eilers-Marx) or the Wood bases.

• If you need help choosing between the two engines, jump straight to the comparison & selection guide.

Runnable companions

Each backend chapter has an executed, self-checking notebook in the repository that follows the same order as the text, formula by formula:

• Part II -> examples/eilers_backend_math_tutorial.ipynb

• Part III -> examples/wood_backend_math_tutorial.ipynb

Every identity in those chapters is recomputed and asserted in the corresponding notebook, and each notebook closes with a pass/fail audit of the head-line checks.