Robustness and speed audit (2026-10-22)¶
An audit of master at d67de78 (1.0.0rc1 and the 26 commits after it) for robustness and speed, in five parts: the C API boundary (A), the execution core (C), the elementwise and row kernels (K), the image, graph and spectral operators (T), and the Python layer (P). Every finding was reproduced. The reproducers are stated inline in each item and become its regression tests. Before the audit the C tests (318 241 checks) and the Python suite (1359 tests) passed: none of these cases was covered.
Each item is a design spec: the problem and its reproducer, the change, how it meets the project's rules (results bit-identical for any thread count, no allocation with a workspace, the ABI, the clean room), its tests, and its acceptance criterion. The items are tracked in ROADMAP.md under "1.0 — audit of 2026-10-22". An item whose design departs from the spec gives the row to add to the decisions table when it lands. Line numbers refer to d67de78, and timings are from an Apple M5 Pro (15 threads) unless an item says otherwise.
Release plan¶
- Correctness, safety and scaling items (target 1.0.0) gate 1.0.0, which is released through a second release candidate, 1.0.0rc2 (A11).
- Speed items that add no API target 1.0.x, after 1.0.0 or before it if ready. Speed items that add API target 1.1.
- Documentation items target 1.0.0.
Decisions that span items¶
- Overlapping outputs (C1, P2). The C library accepts
outidentical to the input: the same data pointer, shape and strides. It returnsFP_EINVALfor any other intersection of the two address ranges, and for a stride-0 output dimension of extent > 1. The rule is conservative: interleaved disjoint views are rejected too, as documented. The Python layer copies an overlapping input first, so Python callers never see that error. - NaN images (A8, P8). On every path (C, PyTorch, JAX) an image that holds
a NaN is not solved. It reports a NaN gap and 0 iterations and counts as
finished, so a NaN never by itself makes a call
FP_ENOTCONV. Retrying with more iterations cannot fix it. - Conjugate prox at small σ (A1). For positively homogeneous g and cone
indicators the conjugate is evaluated as y − prox_g(y), exact for every
σ > 0. The other operators return
FP_EPARAMwhere 1/σ or y/σ overflows. - Power-of-two rescaling (K2, K7–K9, T5, T9). Rescaling happens only when the plain computation leaves a safe range, and by an exact power of two, so results for in-range inputs stay bit-identical.
- Second order in PyTorch (P1). For 1.0.0, the backward raises under
create_graph=True. That rejects some valid parameter-only products, but never returns a wrong Hessian. In 1.0.x it becomes exact where the prox's Jacobian is locally constant, and NaN, never a silent zero, where the curvature is not available.
Decisions on the open questions (2026-10-22)¶
- Versioning (A11): master ships as 1.0.0 through 1.0.0rc2. The "since 1.1" markers are dropped, and the 1.0.0 items land on master once.
- Conjugate-mode duals (A1): accepted. For the homogeneous operators, the
image duals of
prox_conjbecome those of the unit-step problem. The CHANGELOG notes it for warm starts seeded from an earlier conjugate call. - Weighted isotonic (K8): weight ratios min/max below 1e-100 are
rejected with
FP_EPARAM, as for the weighted ℓ1 ball and the hyperplane. There is no fallback path. - PyTorch second order (P1): in two steps. For 1.0.0, the backward raises
a clear error under
create_graph=Trueinstead of returning a wrong Hessian. The full design follows in 1.0.x.
To recheck while implementing:
- C5: Huber's first parameter gradient changes in its last bit once, against 1.0.0rc1, because the old loop got a fused multiply-add.
- C7: one two-thread case measured slower (153 → 173 µs) on a loaded machine. Recheck it on an idle machine before merging.
- C2 notes that 2D TV's per-image working set is about 17 times the image (2.2 GB at 4096²). It is not specified here and is a candidate for a later pass.
New benchmarks¶
The speed criteria name benchmarks that the items add:
bench/bench_python.py:kernel_throughput()(K: row and elementwise kernels, ns per entry);image_tv_derivatives(),edge_tv_batches()andaniso_keep()(T: B1–B3);- three new lines in
overhead(), plustorch_wrappers()andjax_wrappers()(P9–P11). The last two print a message and skip when torch or JAX is missing. bench/profile_kernels.cpp: acore <case>section (C11).
Summary¶
| ID | Item | Kind | Severity | Target | Effort | Depends on |
|---|---|---|---|---|---|---|
| A1 | Conjugate prox at small σ, and steps that overflow inside the kernels | correctness | medium | 1.0.0 | M | none |
| A2 | Shape-product and offset overflow in to_arr / Arr::numel |
safety | medium | 1.0.0 | S | none |
| A3 | Kept calls that do no work leave derivatives unavailable | correctness | low | 1.0.0 | S | A1 (step validation helper step_ok) |
| A4 | Docs: a prox call without keep_structure discards the kept point |
docs | low | 1.0.0 | S | none |
| A5 | Image operators validate and compare fp_options.axis, which they claim to ignore |
correctness | low | 1.0.0 | S | none |
| A6 | An operator must outlive its workspaces (header text) | docs | low | 1.0.0 | S | none |
| A7 | Size errors reported as FP_EINTERNAL (std::length_error) |
correctness | low | 1.0.0 | S | A2 (kMaxElems) |
| A8 | NaN images: the certificate they report, and a dual that cannot be set back | correctness | low | 1.0.0 | S | none |
| A9 | A failed fp_workspace_set_branch treats the earlier override differently by operator kind |
correctness | low | 1.0.0 | S | none |
| A10 | Derivative calls write the "const" workspace: one call per workspace at a time | docs | low | 1.0.0 | S | none |
| A11 | Versioning: "since 1.1" notes on a tree versioned 1.0.0rc1 | docs | low | 1.0.0 | S | none (but decide before tagging 1.0.0) |
| A12 | Enum arguments and fields outside their enumeration (added 2026-10-22) | safety | low | 1.0.0 | S | none |
| C1 | Reject overlapping outputs other than exact aliasing; reject stride-0 outputs | safety | medium | 1.0.0 | S | none |
| C2 | Size per-thread scratch for the threads a row operator can use | scaling | medium | 1.0.0 | S | none |
| C3 | Cap fp_options.threads |
safety | low | 1.0.0 | S | none |
| C4 | Vectorise the elementwise prox loops, contiguous and in place | speed | high | 1.0.x | M | C11 |
| C5 | Elementwise jvp and vjp: register accumulators, branch-free pieces, skip unrequested sums | speed | medium | 1.0.x | M | C4 (same hook style), C11 |
| C6 | Pool: close the job when its pieces are done, instead of waiting for every worker | speed | high | 1.0.x | M | C11 |
| C7 | Grain: pieces scaled to the thread count for prox and jvp | speed | medium | 1.0.x | S | C11 (C6 compounds it) |
| C8 | Strided operator axis: gather and scatter rows in tiles | speed | medium | 1.0.x | M | C2 (scratch accounting), C7 (grain), C11 |
| C9 | Spin time as an environment knob | speed | low | 1.0.x | S | C6 (latency after sleep is then bounded), C11 |
| C10 | Guard against a nested parallel_run from inside a job |
safety | low | 1.0.0 | S | C6 (same file) |
| C11 | A core section in bench/profile_kernels.cpp |
speed (tooling) | low | 1.0.x | S | none |
| K1 | Jacobian 0 instead of 1 where the threshold is 0 and x = 0 | correctness | high | 1.0.0 | S | none |
| K2 | Squared norms overflow or underflow (L2Ball, GroupL2, TreeL2, GroupRadial, SOC) | correctness | high | 1.0.0 | M | none (K5 and K12 build on it) |
| K3 | Log penalty: overflow of (|x|+ε)² and cancellation in the root | correctness | medium | 1.0.0 | S | none |
| K4 | tv1d_l2 derivatives are not scaled, so they fail outside about 1e±103 | correctness | medium | 1.0.0 | M | none |
| K5 | TreeL2 jvp and vjp: τ/s³ goes to inf for groups with ‖w_g‖ < 1e-103 | correctness | medium | 1.0.0 | S | none (shares the scaled norm with K2 if K2 lands first) |
| K6 | Lq: Newton stalls near the threshold (speed) | speed | medium | 1.0.x | S | none |
| K7 | Hyperplane and half-space: Σ aᵢxᵢ overflows | correctness | low | 1.0.0 | M | none |
| K8 | Isotonic: block sums overflow; keep block means instead | correctness (and a small speed gain) | low | 1.0.0 | S | none |
| K9 | TV1D constant_mean overflows |
correctness | low | 1.0.0 | S | none |
| K10 | Huber derivative: c² underflows | correctness | low | 1.0.0 | S | none |
| K11 | A degenerate bound lo = hi is treated as free | correctness | low | 1.0.0 | S | none |
| K12 | L2Ball jvp and vjp: fuse the passes and hoist the division (speed) | speed | low | 1.0.x | S | K2 |
| T1 | Anisotropic TV: flat edges at tiny bounds λ_e | correctness | high | 1.0.0 | M | none |
| T2 | Isotropic TV: threaded derivative solves | speed | high | 1.0.x | L | T3 |
| T3 | Isotropic TV: integer divisions in the derivative loops | speed | medium | 1.0.x | S | none |
| T4 | Graph and tree TV: small batches run serially | speed | medium | 1.0.x | S | none |
| T5 | Image TV: overflow of ‖x‖² and of the gap (power-of-two rescaling) | correctness | medium | 1.0.0 | M | none |
| T6 | Graph TV: dual and gap lost to cancellation at large γ·w | correctness | low-medium | 1.0.0 | M | none |
| T7 | Spectral derivatives: near-ties chained into one wide cluster | correctness | low-medium | 1.0.0 | S | none |
| T8 | Anisotropic TV: serial structure() |
speed | low | 1.0.x | M | T1 |
| T9 | Graph and tree TV: overflow of the value range | correctness | low | 1.0.0 | S | T5 (helper) |
| T10 | Docs: graph TV derivatives with tol > 0 | docs | low | 1.0.0 | S | none |
| P1 | Second-order reverse mode in the PyTorch wrappers (_Prox, _DeviceTV, _SpectralProx) |
correctness | high | 1.0.0 (raise), 1.0.x (full) | L | none |
| P2 | Python side of overlapping out/dx/g buffers |
correctness | medium | 1.0.0 | S | C1 (the C overlap rule) |
| P3 | Serialise calls on one Context | safety | medium | 1.0.0 | S | none |
| P4 | Release the GIL in operator constructors and other data-sized C calls | scaling | low | 1.0.0 | S | none |
| P5 | Keep float32 for non-ndarray array-likes | correctness | low | 1.0.0 | S | none |
| P6 | Reject unknown names and wrong sizes in dparams |
correctness | low | 1.0.0 | S | P1 (derived names via _derived_dparams) |
| P7 | Clear error messages at the Python boundary | correctness | low | 1.0.0 (a), (b), (d); 1.1 (c-parity) | S | P2 (out validation) |
| P8 | Empty batches and NaN or Inf images: one contract across the C, PyTorch and JAX paths | correctness | low | 1.0.0 | M | A8 for the NaN certificate; the empty-batch short-circuits stand alone |
| P9 | JAX: compile once per operator instead of dispatching eagerly | speed | high | 1.0.x | M | none |
| P10 | Keep the derivative structure only when a derivative can follow (PyTorch and JAX forward) | speed | medium | 1.0.x | S | P1 (touches _Prox.forward) |
| P11 | Trim per-call Python overhead | speed | low | 1.0.x | S | P3 (the lock adds ~0.05 µs) and P2 (adds 0.08 µs); the targets include both |
| P12 | Free-threaded CPython (3.13t, 3.14t) | docs | low | 1.1 (wheels), 1.0.0 (a docs line) | M | P3 (required), P4 |
A. C API boundary¶
Scope. The extern "C" boundary: src/api/api.cpp, include/fastprox.h, src/core/workspace.hpp, and the
hand-off to src/core/driver.hpp. Kernel internals, the Python layer, aliasing and overlap of outputs, and the
thread cap are covered elsewhere.
Notation.
- Line numbers are those of master at d67de78.
- Reproducers were run against an ASan+UBSan Debug build (-DFASTPROX_SANITIZE=ON -DCMAKE_BUILD_TYPE=Debug). Each one is stated inline: calls, shapes and values. Unless stated otherwise: one-shot calls (opt = NULL, ws = NULL), C-contiguous float64 arrays.
- The survey grid (A1) runs these operators, as one row of length 6 with x = [0.5, 2, −3, 1, 20, −7] (4×4 images with x = [0.5, 2, −3, 1, 20, −7, 0.3, −0.2, 4, 1.5, −2.5, 0.1, 9, −1, 3, 0.7] for anisotropic and isotropic TV), with opt.tol = 1e-12 and opt.max_iter = 100000:
- soft(λ = 10), elastic net(1, 10), Huber(10, 1), group ℓ2(10), tree ℓ2(0.5, one group);
- box[−1, 2], nonneg, log barrier(10), ℓ1-box(1, [−1, 2]);
- ℓ2 ball(1), simplex(1), ℓ1 ball(1);
- ℓ∞(10), max(10), sum of the 2 largest(10), plain and absolute;
- hyperplane(a = 1, b = 1), half-space with box(a = 1, b = 1, [−1, 2]);
- SOC, isotonic, tv1d, TV ball(1), tv1d_l2;
- ℓ0(1), ℓq(1, 0.5), SCAD(1, 3.7), MCP(1, 3), log penalty(1, 0.1), group Huber(1, 1);
- graph TV and tree TV on the chain 0–1–…–5;
- anisotropic and isotropic TV in 2D.
Each runs with fp_prox_conj at σ ∈ {1e-10, 1e-100, 1e-300, 1e-307, 1e-308, 6e-309, 5e-309, 1e-320} and with fp_prox at γ ∈ {1e100, 1e154, 1e200, 1e300, 1e308, DBL_MAX}, in float64 and float32. "Off by d" compares with fp_prox_conj at σ = 1.
Constants introduced here, and shared by A1, A2 and A7:
// src/core/layout.hpp
constexpr int64_t kMaxElems = INT64_MAX / 8; // 2^60 - 1: numel * itemsize and every byte offset fit int64
constexpr double kMaxScaledStep = 0x1p960; // ≈ 2.4e289: headroom 2^64 for sums of step-scaled terms
A1. Conjugate prox at small σ, and steps that overflow inside the kernels¶
- Kind: correctness · Severity: medium · Target: 1.0.0 · Effort: M · Depends on: none
Problem. prox_impl (api.cpp:266, 336) checks only that the user's step is finite. For fp_prox_conj the kernel runs at γ = 1/σ on y/σ, so the results go wrong in three ways:
- 1/σ overflows for σ < 1/DBL_MAX ≈ 5.6e-309. Soft thresholding with λ = 10 and y = [0.5, 2, −3, 1] then returns
z=[nan nan nan nan]withFP_OK; the expected result is y clipped to [−10, 10]. Row operators (tv1d, ℓ∞(10), group ℓ2(10)) returnFP_EINVALfor the same call. - y/σ overflows the dtype. On the survey grid in float32, every operator fails from σ = 1e-100 already: elementwise ones give NaN with
FP_OK, row onesFP_EINVAL. - Silent wrong results while 1/σ is still finite. In float64 at σ = 1e-300, the following return wrong values with
FP_OKbecause of squares or sums of the huge y/σ: - group ℓ2: off by 9.3;
- tree ℓ2: off by 0.94;
- anisotropic TV: off by 2.0;
- isotropic TV: off by 3.2;
- SOC and group Huber: NaN.
Separately, a huge but finite step γ in fp_prox overflows γ·λ inside some kernels (survey grid, float64):
- log barrier (λ = 10) at γ = 1e308 on x = [0.5, 2, −3, 1] gives
[inf inf nan inf]; the exact result is about 3e154; - max and sum of the k largest give −inf where the exact result, −1.7e308, is representable;
- sum of the k largest absolute values gives NaN;
- tree TV at γ = DBL_MAX gives z₀ = 5.99e307 instead of the mean 2.25.
Design. Three parts. Elementwise and row operators share every path, so their statuses agree by construction.
A1.1 — operators whose g is positively homogeneous (exact for every σ > 0).
- Identity. If g(tz) = t g(z) for t > 0 (norms, seminorms, the maximum, the sums of the largest entries, every TV), or g is the indicator of a cone, then prox_{γg}(tx) = t·prox_{(γ/t)g}(x). With γ = t = 1/σ this gives
y − σ prox_{g/σ}(y/σ) = y − prox_{g}(y).
g* is then the indicator of a closed convex set C, and the result is the projection onto C, the same for every σ > 0. This is still the Moreau identity of spec Section 4, evaluated without scaling.
- New virtual. Add virtual bool conj_homogeneous() const noexcept { return false; } to fp_op (op.hpp, next to check_step).
- Overrides returning true:
- Soft, GroupL2 (group ℓ2 and ℓ2), TreeL2 (tree and sparse group), LInf, Max, SumLargest (both modes);
- TV1D, TV1DL2, TVAniso (plain and weighted), TVIso (plain and weighted), and edge_tv::Base (graph TV and tree TV);
- the cone indicators NonNeg, Soc and Isotonic (weighted too, since the weighted projection onto a cone is positively homogeneous).
- Not included (optional later): cases that are homogeneous only for some parameters (elastic net with l2 = 0, a box or hyperplane with b = 0 and every bound in {−∞, 0, +∞}).
- Change in prox_impl (api.cpp:336-337):
const bool homog = conj && op->conj_homogeneous();
const double gamma = !conj ? step : homog ? 1.0 : 1.0 / step;
const double sigma = !conj ? 0.0 : homog ? 1.0 : step;
c.gap·σ² is unchanged in meaning: the scaled problem is σ⁻² times the unit one.
- Workspace state. Add bool conj_homog to fp_workspace (workspace.hpp, next to conj). Set it where ws->conj is set (api.cpp:359, 373).
- Derivatives. The map is constant in σ > 0. In jvp_impl, pass dgamma = 0.0 to the drivers when ws->conj_homog. In vjp_impl (api.cpp:523-527), when ws->conj_homog:
- set tot[P] = 0.0 exactly instead of tot[P + 1] + tot[P] / ws->sigma;
- keep the θ rule tot[k] *= -ws->sigma (σ = 1), which is the exact −(∂prox_g(y)/∂θ)ᵀg.
- Visible change (image operators). After fp_prox_conj, the workspace dual becomes the dual of prox_g(y) at unit step: σ times the old one, the same for every σ, and the conjugate output equals Σ_a q_a. This makes warm starts across changing σ (PDHG with adaptive steps) consistent. It goes in the CHANGELOG.
A1.2 — the other operators: the formula as written, with a checked range.
- Which operators: ElasticNet, Huber, Box, LogBarrier, L1Box, L2Ball, Simplex, L1Ball, HyperBox, TVBall, GroupRadial, L0, Lq, Scad, Mcp, LogPen.
- σ too small. In prox_impl, after computing gamma: if (conj && !homog && !std::isfinite(gamma)) return FP_EPARAM;.
- y/σ overflows. Check while staging y/σ; the scaled entry must be finite in the array's dtype, else the call returns FP_EPARAM. Three sites:
- elementwise in ElemOp::run_prox (op.hpp:368-376). Add a trailing bool *overflow parameter to both fp_op::prox_run overloads, to run_prox and to FP_ELEM_FORWARD. In the conjugate loop, set *overflow = true when std::isfinite(y) && !(std::fabs(double(t)) <= std::numeric_limits<T>::max()). prox_driver (driver.hpp:131-143) ORs a per-chunk flag into a std::atomic<bool>;
- rows in prox_driver (driver.hpp:168-170), the same test into the same atomic;
- images in image_prox_driver (driver.hpp:585-586). No image operator takes this path after A1.1; the check is defence for future operators.
- Status after the loop. The driver returns FP_EPARAM if the flag is set (before the err test, which stays FP_EINVAL). Every row and chunk is still processed, so the status does not depend on the thread count.
- NaN and ±Inf in y keep today's behaviour: a NaN propagates, and ±Inf is operator-defined.
A1.3 — a step range for every call.
- New virtual. Add virtual double step_scale() const noexcept { return 1.0; } to fp_op: an upper bound on the factors by which the kernel multiplies its step.
- Overrides:
- Soft, L1Box, L0, Lq, Scad, Mcp: λ·max w;
- ElasticNet: max(l1, l2);
- Huber: λ·max(1, 1/δ);
- LogBarrier, LInf, Max, SumLargest: λ;
- LogPen: λ·max(1, 1/ε);
- GroupL2: λ·max c;
- TreeL2: max(l1, max η);
- GroupRadial: inner step_scale()·max c;
- TV1D, TV1DL2, edge_tv::Base: max w;
- TVAniso: maxₐ wₐ·max β;
- TVIso: maxₐ wₐ·max α;
- Box, NonNeg, L2Ball, Simplex, L1Ball, HyperBox, Soc, Isotonic, TVBall: 0 (γ is ignored).
- Helper in api.cpp, used at api.cpp:344 (each step of fp_prox_steps) and api.cpp:348:
fp_status step_ok(const fp_op &op, double g) noexcept {
const double s = op.step_scale();
if (s > 0.0 && !(g * s <= kMaxScaledStep)) return FP_EPARAM;
return op.check_step(g);
}
hypot fallback (pointwise.cpp:284-285) handles x² overflow, since 4γλ ≤ 2⁹⁶² stays finite.
- Kernels. None needs a new formula inside the range. The acceptance test checks this on the survey grid. A kernel that still fails is fixed in the kernel, not by tightening the bound.
As implemented.
- The float64 overflow check runs only for σ < 2⁻⁶⁴ (decided 2026-10-22). A check for every σ < 1 cost 15–30% on cheap conjugates such as the box. Separate passes and checks folded into the loop measured the same. Above 2⁻⁶⁴, float64 entries beyond about 1e289 can overflow, and their outputs are then ±inf or NaN. float32 is checked for every σ < 1, since there the conversion would be undefined.
- No signature change. Instead of a new prox_run parameter, run_prox returns −1 when a quotient overflowed; tie counts are never negative. Rows use a new status bit, kRowStepOverflow. The check is a division-free pass, staging_overflows, that compares bit patterns as integers.
- Huber and the log penalty. Their step_scale() is λ, not λ·max(1, 1/δ) or λ·max(1, 1/ε): since K10 their kernels form only γλ, and 1/δ overflowed for subnormal δ.
- Tests. The non-homogeneous operators at σ = 1e-300 return FP_EPARAM wherever 1/σ = 1e300 leaves the step range; only the projections, which ignore the step, return FP_OK. The existing tests with steps beyond the range were updated:
- isotropic TV at γw = 1e300;
- the log barrier, max and the sums of the largest entries at 1e300;
- the homogeneity tests at 2¹⁰⁰⁰, now at 2⁹⁰⁰.
Contract text.
- fastprox.h, fp_prox: "gamma must be finite, ≥ 0, and at most 2^960 (≈ 2.4e289) divided by the operator's largest parameter (λ, a weight, 1/δ, ...; projections ignore gamma); FP_EPARAM otherwise."
- fastprox.h, fp_prox_conj (replaces the one-line comment):
/* prox_{sigma g*}(y) = y - sigma prox_{g/sigma}(y/sigma), sigma >= 0; sigma = 0
* returns y. When g is positively homogeneous (soft thresholding, group l2,
* tree l2, l-inf, max, the sums of the largest entries, every TV operator) or
* the indicator of a cone (nonnegative orthant, second-order cone, monotone
* cone), g* is the indicator of a set C and the result, y - prox_g(y), is the
* projection onto C: the same for every sigma > 0, computed without scaling,
* and its derivative in sigma is 0. For the other operators the formula is
* evaluated as written: 1/sigma must satisfy fp_prox's step range and every
* y_i / sigma must be finite in the array's dtype, else FP_EPARAM. */
fp_prox_conj the dual is that of prox_g(y) and Σₐ qₐ is the output". Add to the FP_EPARAM row of the error table: "a step outside the operator's range, or a σ too small for the conjugate formula".
Alternatives considered. - Returning y for tiny σ: wrong for homogeneous g. Soft with y = 20 and λ = 10 must give 10 at every σ. - A pre-pass for max|y| per call: one more read of the array on the hottest conjugate path. - Per-operator stable conjugate forms for set indicators (y − P_{σC}(y), with the set's parameters scaled by σ): exact, but they need new σ-derivative formulas in nine kernels, for a regime (σ < 5.6e-309, or float32 |y|/σ > 3.4e38) with no known use.
Constraints.
- Determinism: unchanged. The overflow flag is an OR over all chunks and rows, computed in full for any thread count.
- Allocation: none. No new workspace bytes.
- ABI: none. Internal virtuals and fields only; prox_run's signature is internal.
- Clean room: not affected, no external source.
- Results: homogeneous operators change at the rounding level for σ ≠ 1 (y − prox_g(y) instead of the scaled form). This goes in the CHANGELOG.
- Spec departure: yes, a new documented step range. Decisions row: | 2026-10-22 | fp_prox_conj evaluates y − prox_g(y) for positively homogeneous g and cone indicators (exact for every σ > 0, σ-derivative 0, image duals at unit step); for the other operators it returns FP_EPARAM when 1/σ or some y_i/σ overflows. Every call returns FP_EPARAM when γ·step_scale() > 2^960 | src/api/api.cpp, src/core/op.hpp, src/core/driver.hpp, src/ops/*, fastprox.h, docs/api.md |
Tests.
- tests/c/test_core.cpp, new test_conj_extremes():
- homogeneous operators: every A1.1 operator, float32 and float64, the survey grid's inputs (rows of length 6, 4×4 images). For σ ∈ {5e-324, 1e-310, 1e-300, 1e-100, 1, 1e100, 1e300}: fp_prox_conj(σ) is bitwise equal to fp_prox_conj(1.0). Soft (λ = 10) on y = [20, −3, 0.5] gives exactly [10, −3, 0.5] at σ = 1e-320 and at σ = 1e300.
- derivatives: with keep_structure, fp_jvp with dgamma = 1 is bitwise equal to the one with dgamma = 0, and fp_vjp's *ggamma == 0.0.
- other operators: every A1.2 operator returns FP_EPARAM at σ = 1e-310. In float32 with y = 1e10 and σ = 1e-30, elastic net (elementwise) and the ℓ2 ball (row) both return FP_EPARAM. In float64 at σ = 1e-300 on the survey grid's inputs: FP_OK and finite.
- step range: at γ = 1e308 and λ = 10, the log barrier, max, sum of the largest and tree TV return FP_EPARAM.
- inside the range: at γ = 1e280 the outputs are finite, the log barrier at x = −3 matches (x + hypot(x, 2√γ√λ))/2 to 1e-15 relative, and tree TV on a chain equals the mean of x to 1e-15·max|x|.
- tests/python/test_operators.py::test_moreau_conjugates: widen sigma to st.floats(1e-320, 1e300) for Soft, GroupL2 and LInf (homogeneous: same tolerances).
- fuzz/fuzz_ops.cpp: replace the exclusion !(conj && gamma < 1e-6) and gamma < 1e6 by "every finite step". The new invariant is that a successful prox on finite moderate input has no NaN in its output (±inf stays allowed for genuine float32 overflow).
Acceptance.
- Correctness: test_conj_extremes, the widened Hypothesis test and 10 minutes of the standalone fuzzer pass. Re-running the survey grid gives no non-finite output with FP_OK, except float32 entries whose exact value exceeds FLT_MAX (log barrier, max, sum of the largest at γ ≥ 1e100).
- Speed: uv run python bench/bench_python.py shows no regression. Conjugate calls of homogeneous operators lose one division per entry.
A2. Shape-product and offset overflow in to_arr / Arr::numel¶
- Kind: safety · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: none
Problem. to_arr (api.cpp:54-67) and Arr::numel() (layout.hpp:29-33) multiply extents in unchecked int64, and nothing bounds the element offsets that the strides reach. Confirmed cases:
- shape (2³², 2³²) with data = NULL: UBSan reports signed overflow, then fp_prox returns FP_OK;
- shape (2³²+1, 2³²) on a 64-element buffer: fp_workspace_create returns FP_OK, then fp_prox reads out of bounds (ASan stack-buffer-overflow in ElemOp<Soft>::run_prox, op.hpp:386);
- tv1d on (3, 2⁶²): the same;
- shape (2⁶²+1, 4) with strides (0, 1) and keep_structure: numel wraps to 4, build_workspace sizes ws.code for 4 entries, and the row loop writes 2⁶²+1 rows into it (heap overflow).
That last descriptor addresses only 4 elements of memory, so a caller can produce it legitimately, for example with a broadcast view.
Design. - Checked multiply in layout.hpp:
// r = a * b for 0 <= a, b <= kMaxElems; false when the product exceeds kMaxElems.
inline bool mul_bounded(int64_t a, int64_t b, int64_t &r) noexcept {
if (a != 0 && b > kMaxElems / a) return false;
r = a * b;
return true;
}
to_arr, after reading the shape and strides (NULL strides computed as now):
1. Count: count = 1. For each d with shape[d] > 1, mul_bounded(count, shape[d], count); any failure → FP_ESHAPE. Zero extents are skipped, so every prefix product that Arr::numel() later forms is at most kMaxElems, even for (2⁴⁰, 2⁴⁰, 0), which is rejected.
2. Span: for each d with shape[d] > 1, require -kMaxElems <= strides[d] <= kMaxElems (this also rejects INT64_MIN). Accumulate span += (shape[d] - 1) * |strides[d]| through mul_bounded, with span <= kMaxElems at every step; failure → FP_ESHAPE. Every element offset then lies in [−span, span], so offsets times the item size (api.cpp:325, the drivers) and the pointer arithmetic stay in int64.
3. NULL strides: the computed C strides are at most count and need no extra check. Run the span check on the computed strides anyway, which keeps the code uniform.
Every public array goes through to_arr: x, out, tangents, cotangents, steps, masks, sides, duals, parameter vectors, fp_value's out, the spectral a, b, s, z and r. Fixing it there covers all of them.
- As implemented: row_thread_bytes returns SIZE_MAX for rows over 2⁴⁸ entries: its 4·n·item and the kernels' scratch_bytes(n) could wrap before any product, and only stride-0 views reach such rows. The image sizes need no cap, since check_image limits an image to 2³¹ − 1 entries.
- build_workspace (api.cpp:159-235): compute every byte count with a saturating size_t mul_size(size_t a, size_t b) (SIZE_MAX on overflow). This covers image_bytes * image_sets, image_thread_bytes * nthreads, B * k * N, numel * item, struct_bytes * B, thread_bytes * nthreads and max(numel, B * kink_sites). A saturated size makes Buffer::resize call new T[SIZE_MAX / 8 + 1], which throws std::bad_array_new_length (a std::bad_alloc). That becomes FP_ENOMEM, never a short buffer.
- Header text (fastprox.h, fp_array comment): "The product of the extents, ignoring zero extents, and the largest element offset, Σ_d (shape[d] − 1)·|strides[d]|, must not exceed 2^60 − 1; FP_ESHAPE otherwise. ndim may be 0 (a scalar)." The same sentence goes in docs/api.md under the fp_array block.
- Alternatives considered.
- __builtin_mul_overflow: not available on MSVC, which the wheels build with.
- Bounding the count by the stride footprint only: wrong, because the batch loops iterate the logical count (the (2⁶²+1, 4) case).
Constraints. - Determinism: not affected. - Allocation: none on the call path; validation is O(ndim). - ABI: none. - Spec departure: no. Spec Section 9 already promises validation at the boundary.
Tests.
- tests/c/test_core.cpp, test_errors_and_special_values(). Each of the following returns FP_ESHAPE from fp_workspace_create, fp_prox, fp_prox_conj and fp_value:
- (2³², 2³²) with data NULL;
- (2³²+1, 2³²) on a 64-element buffer;
- (3, 2⁶²);
- (2⁶²+1, 4) with strides (0, 1) and keep;
- (2⁴⁰, 2⁴⁰, 0);
- (2, 4) with strides (INT64_MIN, 1);
- (2, 2) with strides (2⁶⁰, 1).
- Still FP_OK: (1000, 4) with strides (0, 1), (0, 5) and (5, 0).
- fuzz/fuzz_ops.cpp: in one case in 64, set shape[0] from a raw u64() with stride 0 and check that the status is valid, with no ASan report.
Acceptance. Correctness: the new checks and the existing suite pass under the FASTPROX_SANITIZE=ON tree, and the first three reproducers of the Problem return FP_ESHAPE. Speed: not affected (O(ndim) per call).
A3. Kept calls that do no work leave derivatives unavailable¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: A1 (step validation helper
step_ok)
Problem. Two early returns in prox_impl skip setting ws->valid:
- if (x.numel() == 0) return FP_OK; (api.cpp:309);
- the σ = 0 identity path of fp_prox_conj (api.cpp:317-330).
So fp_jvp and fp_vjp return FP_EINVAL after a successful kept call:
- tv1d with keep_structure on x of shape (0, 5) (data NULL): fp_prox returns FP_OK, fp_jvp and fp_vjp FP_EINVAL;
- soft(1) with keep_structure on y = [0.5, 2, −3]: fp_prox_conj at σ = 0 returns FP_OK, then fp_jvp FP_EINVAL. fp_prox at γ = 0 followed by fp_jvp gives FP_OK. The ws->numel == 0 branches of jvp_impl (api.cpp:423), vjp_impl and kinks_impl can never run. The empty-batch return also comes before every step check. An empty x accepts a SCAD step that breaks a > 1 + γ, and fp_prox_steps on x of shape (3, 0) accepts steps [1, −1, 2], although the header says "every step is checked before any work".
Design.
- Empty batches. In prox_impl, move the step validation (step_ok for γ; for fp_prox_steps, every step) before the numel == 0 return. Loop over the steps array's own count, the product of steps.shape (which equals ws->rows when numel > 0), instead of over ws->rows, which is 0 for shape (3, 0). Then, if x.numel() == 0: when keep, set ws->valid = true and conj, conj_homog, gamma, sigma and tie as on the normal path, with steps_on = false, and return FP_OK. The existing numel == 0 branches then give:
- fp_jvp: FP_OK, nothing written;
- fp_vjp: FP_OK, zero gparams and *ggamma = 0;
- kinks: count 0.
- σ = 0 in fp_prox_conj (the result is y exactly). Add bool identity to fp_workspace. When keep, set valid = true, conj = true, identity = true and sigma = 0; every other prox call clears identity. Then:
- jvp_impl: dout = dx (zeros when dx is NULL), copied through the strides with the loop of api.cpp:318-328, factored into a helper copy_array(src, dst). When dgamma != 0, write NaN to every entry instead. dparams contributes nothing.
- vjp_impl: gx = g, gparams = 0, *ggamma = NaN.
- fp_workspace_kinks: count 0 and a zero mask, written through the mask's strides.
- fp_workspace_set_branch: FP_OK with no effect.
NaN rather than 0 for the σ-derivative: the right derivative at σ = 0 is −∇g*(y) where g* is smooth (elastic net: −y/l2), and it is undefined where the conjugate's domain is bounded (every homogeneous g jumps from y to P_C(y)). Reporting 0 would be silently wrong.
- Contract text.
- fastprox.h, fp_prox_conj: "sigma = 0 returns y. With keep_structure, the derivatives are those of the identity: J = I and zero parameter gradients. The sigma-derivative, which is not defined there for every operator, is NaN (fp_vjp's ggamma; fp_jvp's output when dgamma ≠ 0)."
- fastprox.h, the entry-points block: "An empty batch with keep_structure saves an empty point: derivatives return FP_OK and write nothing, gradients 0."
- docs/api.md, "### Conjugates": the same text.
- Alternatives considered.
- Keeping FP_EINVAL after these calls and documenting it: autograd wrappers would then special-case empty batches and σ = 0.
- A σ-derivative of 0 at σ = 0: wrong for smooth conjugates, as above.
Constraints.
- Determinism: not affected.
- Allocation: none; identity is a bool.
- ABI: none.
- Spec departure: yes (the σ = 0 derivative). Decisions row: | 2026-10-22 | After fp_prox_conj with σ = 0 and keep_structure, derivatives are those of the identity and the σ-derivative is NaN; kept empty batches give empty derivatives | src/api/api.cpp, fastprox.h, docs/api.md |
Tests. In tests/c/test_core.cpp, test_errors_and_special_values():
- Empty batches: soft, tv1d and tv_iso on x of shape (0, 5), (3, 0) and (0, 4, 4) with keep: prox, jvp and vjp return FP_OK; gparams are all 0, *ggamma == 0 and the kink count is 0.
- σ = 0: soft, l2_ball and tv_iso with fp_prox_conj(σ = 0) and keep:
- the output is bitwise y;
- jvp with dgamma = 0 gives dout bitwise equal to dx, and with dgamma = 1 gives all-NaN;
- vjp gives gx bitwise equal to g, gparams 0 and ggamma NaN.
- Step checks on empty x: SCAD (a = 3.7) on an empty x with γ = 10 returns FP_EPARAM. fp_prox_steps with x of shape (3, 0) and steps [1, −1, 2] returns FP_EPARAM.
As implemented.
- The design is followed throughout, with copy_array (api.cpp) shared by the identity's jvp, vjp and kink mask.
- fp_workspace_set_branch on an empty point returns FP_OK with no effect, like the identity; the fuzzer reached it once empty points were valid.
- The image branch of build_workspace returned before sizing tot and dparams for an empty batch, so a kept empty fp_vjp stored through a null pointer. It now sizes those two buffers.
- copy_array skips empty arrays, because for_runs divides by every extent.
- σ = 0 on an empty batch saves the identity, so its σ-derivative is NaN as well.
Acceptance. Correctness: the tests above pass, and both reproducers of the Problem return FP_OK from every call. Speed: not affected.
A4. Docs: a prox call without keep_structure discards the kept point¶
- Kind: docs · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. docs/api.md:697 says derivatives use "the structure saved by the last prox call that had keep_structure set", and spec.md:514 and fastprox.h:367 and 461 say much the same. The code (api.cpp:292) invalidates the point on every prox call on the workspace. Reproducer: soft(1) on y = [0.5, 2, −3] with a kept workspace. A kept fp_prox (γ = 1) is followed by fp_jvp → FP_OK; then fp_prox without keep on the same workspace, then fp_jvp → FP_EINVAL. The code is right, since a call without keep overwrites the image duals and the scratch the derivatives read, so the docs must change.
Design. Text only.
- docs/api.md:697-698: "Derivatives use the point saved by the last prox call on the workspace (fp_prox, fp_prox_steps or fp_prox_conj), which must have had keep_structure set. Any later prox call on the workspace replaces that point, or discards it when the call has no keep_structure or fails; derivatives then return FP_EINVAL until the next kept call. To keep a point while computing other proxes, use a second workspace."
- fastprox.h:460-462 (fp_jvp): "at the point saved by the last prox call on ws (fp_prox, fp_prox_steps or fp_prox_conj), which must have had keep_structure; a later call without it, or one that fails, discards the point (FP_EINVAL)."
- fastprox.h:367-368 (kinks): "at the point saved by the last prox call on ws, if it had keep_structure (FP_EINVAL otherwise, and for other operators)".
- spec.md:514: "read the structure saved in the workspace by the last prox call on it, which must have had keep_structure set".
- Check the Python docstrings that mention keep= and ctx for the same wording.
Constraints.
- Behaviour: no code change.
- Spec departure: the spec's wording was ambiguous and is corrected in place. Decisions row: | 2026-10-22 | A prox call without keep_structure, or a failed one, discards the workspace's kept point (the spec read "the last call with keep_structure") | docs/spec.md, docs/api.md, fastprox.h |
Tests. tests/c/test_core.cpp: lock the behaviour with the reproducer. Keep prox, then jvp FP_OK; a prox without keep, then jvp FP_EINVAL; keep prox again, then jvp FP_OK. Also: a kept prox, then a failed kept prox (γ = −1, FP_EPARAM), then jvp FP_EINVAL.
As implemented. The texts as designed, with one change found while writing the test: the code did not discard the point on every failed call. A negative step and an overlapping output returned before the workspace was touched, but a step beyond the step range, a bad tol and a length the operator rejects returned after it had been reset. prox_impl now resets the workspace only after every argument check, so a call rejected for its arguments leaves the point, and the texts say so; a call without keep, or one that fails during its work, discards it. test_kept_point_lifetime in tests/c/test_core.cpp checks the reproducer, four rejected calls (negative step, step out of range, wrong output shape, negative tol), the discarding calls (without keep, and the conjugate at σ = 0) and a new kept call, for soft thresholding and anisotropic TV; it fails 10 checks on the old code. Decisions row of 2026-10-23.
Acceptance. The test passes and the four texts agree.
A5. Image operators validate and compare fp_options.axis, which they claim to ignore¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. The header says image operators (TV in 2 and 3 dimensions, graph TV, tree TV) act on the last axes and that fp_options.axis is ignored. But prox_impl (api.cpp:275) and fp_workspace_create (api.cpp:1269) normalise it, and prox_impl compares it with ws->axis (api.cpp:287). Reproducer: tv_aniso(2) on a 4×4 image. fp_prox with opt.axis = 5 → FP_EINVAL. A workspace created with opt = NULL (axis −1), then fp_prox with opt.axis = 0 → FP_EINVAL.
Design.
- prox_impl and fp_workspace_create:
int axis;
if (op->image_ndim()) axis = x.ndim - 1; // canonical; the option is ignored
else if (norm_axis(o.axis, x.ndim, axis)) return FP_EINVAL;
ws->axis comparison always passes for them. x.ndim >= 1 after to_arr; the existing x.ndim < k checks stay where they are. read_steps and fp_value already ignore the axis for image operators.
- Docs: none to change; the header becomes true. Optionally add "any value, unchecked" next to "fp_options.axis is ignored" at fastprox.h:267 and 307.
Constraints. No effect on determinism, allocation or ABI, and no spec departure (the code now matches the 2026-10-16 decisions row).
Tests. tests/c/test_core.cpp, for tv_aniso, tv_iso and graph_tv:
- fp_prox with axis 5 and with axis 0 is bitwise equal to axis −1;
- a workspace created with axis 7 accepts calls with axis −1;
- a workspace created with axis −1 and keep accepts a call with axis 0, after which fp_jvp returns FP_OK.
As implemented. As designed, in both prox_impl and fp_workspace_create.
Acceptance. The tests pass and both calls of the reproducer return FP_OK.
A6. An operator must outlive its workspaces (header text)¶
- Kind: docs · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. A workspace keeps a raw const fp_op * (workspace.hpp:52). kink_arrays (api.cpp:552), set_branch_impl, apply_structure and dual_impl dereference it without the caller's operator. Nothing in fastprox.h or docs/api.md says the operator must stay alive. Reproducer: soft(1) on y = [0.5, 2, −3]. A kept fp_prox, then fp_op_free(op), then fp_workspace_kinks(ws, 0, NULL, &count) gives an ASan heap-use-after-free at api.cpp:552.
Design.
- fastprox.h, next to fp_op_free: "Free an operator only after every workspace created for it: workspaces refer to their operator, and fp_workspace_kinks, fp_workspace_set_branch and the other workspace functions read it."
- fastprox.h, fp_workspace_create: "op must outlive the workspace."
- docs/api.md, "### Lifecycle": after "Operators are immutable and can be shared between threads.", add "An operator must stay alive until every workspace created for it has been freed."
- Alternative considered: reference counting (an atomic count in fp_op, incremented by fp_workspace_create, with the delete deferred to the last release). It is ABI-neutral, since the handles are opaque, and would also stop a freed operator's address from being reused while a workspace still holds it. Deferred because it changes when fp_op_free releases memory and costs an atomic per workspace. It can come in 1.x without an ABI change if real code trips on this.
Constraints. Text only: no behaviour, allocation or ABI change, and no spec departure.
Tests. None: the misuse is undefined behaviour. The doc build checks the text.
As implemented. As designed. The header's workspace block gained the sentence too.
Acceptance. The three texts are present.
A7. Size errors reported as FP_EINTERNAL (std::length_error)¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: A2 (
kMaxElems)
Problem. guard (api.cpp:26-34) maps std::bad_alloc to FP_ENOMEM and every other exception to FP_EINTERNAL. A std::vector larger than max_size() throws std::length_error, so a size problem is reported as an internal error. Reproducer: fp_op_soft(1.0, &w, &op) with w a float64 array of shape (2⁶²) and stride 0 over one double → FP_EINTERNAL. After A2 such an array fails earlier with FP_ESHAPE, but the raw counts below still reach std::vector::assign unchecked:
- n of fp_op_group_l2 and of the group Huber/SCAD/MCP constructors (api.cpp:919, 858);
- ngroups and group_ptr[ngroups] of fp_op_tree_l2 (api.cpp:960, 965).
Forming groups + n for such an n is also undefined behaviour before any allocation.
Design.
- Bound the counts before forming pointers:
- group constructors: in fp_op_group_l2 and group_radial, return FP_EPARAM when n > kMaxElems;
- fp_op_tree_l2: return FP_EPARAM when ngroups > kMaxElems - 1 or ptr.back() > kMaxElems.
Graph and tree TV already bound n and m (edge_tv::parse, edge_tv.hpp:43).
- Defence in depth in guard: add catch (const std::length_error &) { return FP_ENOMEM; } after the bad_alloc handler.
- Contract text:
- fastprox.h:67: FP_ENOMEM, /* out of memory, or a size too large to allocate */;
- docs/api.md, error table: "out of memory, or a requested size too large to allocate";
- spec.md, Section 9 "Boundary pattern": "std::bad_alloc and std::length_error become FP_ENOMEM".
- Alternative considered: mapping only, without the bounds. That leaves the pointer-arithmetic undefined behaviour in place.
Constraints.
- Determinism, allocation, ABI: not affected.
- Spec departure: yes (boundary pattern). Decisions row: | 2026-10-22 | std::length_error is reported as FP_ENOMEM (spec: any exception other than bad_alloc is FP_EINTERNAL); integer counts of the group constructors above 2^60 − 1 are FP_EPARAM | src/api/api.cpp, fastprox.h, docs/spec.md |
Tests. tests/c/test_core.cpp, test_errors_and_special_values():
- fp_op_group_l2(1.0, INT64_MAX / 2, groups, 4, NULL, &op) with a 4-entry groups → FP_EPARAM, and op == NULL;
- fp_op_tree_l2 with group_ptr = {0, INT64_MAX / 2} → FP_EPARAM;
- soft with weights of shape (2⁶²) and stride 0 → FP_ESHAPE (A2).
As implemented.
- As designed, in group_radial (group Huber, SCAD and MCP) and fp_op_group_l2.
- fp_op_tree_l2 also bounds ngroups before forming group_ptr + ngroups + 1.
- The guard mapping cannot be reached through a test once the bounds are in, so it stays as defence in depth.
Acceptance. The tests pass and no constructor returns FP_EINTERNAL on these inputs.
A8. NaN images: the certificate they report, and a dual that cannot be set back¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. An image containing a NaN gives a NaN output with FP_OK, as documented. Two things around it are wrong.
- Certificate. The kernels' NaN branches return before solving, with
c.gap = 0,c.iters = 0andc.converged = true(tv_aniso.cpp:252-254 and 264, tv_iso.cpp:302-305,edge_tv::Base::read_input, edge_tv.hpp:117-126). So a NaN image reports a perfect certificate. - Reproducer: batch of two images, tv_aniso(2) and tv_iso(2) on (2, 4, 4), graph_tv and tree_tv on the chain 0–1–…–15 with x of shape (2, 16); x_i = 3 sin(1.7 i), γ = 0.7,
opt.tol = 1e-10,opt.max_iter = 5000. With every entry NaN, all four returnFP_OKwithgap=0.itersis 0, except tree TV, which reports 1. - Device path: it iterates to
max_iteron such an image and reports gap NaN. So the C and device certificates disagree, andIterInfosaysconverged=True, gap=0for an image that was never solved. - Dual round trip. The same branches fill the image's dual with NaN on purpose (tv_aniso.cpp:266, tv_iso.cpp:307, edge_tv.hpp:126), and
dual_validstays true.fp_workspace_get_dualthen returnsFP_OKwith NaN entries, whichfp_workspace_set_dualrejects (api.cpp:835,FP_EINVAL), so get → set fails. - Reproducer: tv_iso(2) on one random 16×16 image,
fp_workspace_set_warm(ws, 1); a clean prox, then a prox with one NaN entry, thenfp_workspace_get_dual→FP_OKwith all 512 entries NaN. The next warm call on the clean image recovers (finite z), because the warm-start pass already sets non-finite entries to 0.
Design.
A8.1 — the certificate of a NaN image (tv_aniso, tv_iso, graph_tv, tree_tv; C and device paths alike).
- Rule: a NaN image is not solved. It reports gap = NaN, iterations = 0, and it does not count as stopped before the tolerance: it never makes the call FP_ENOTCONV by itself. The batch's fp_info.gap, the sum over images, is then NaN. fp_info.iters, the maximum over images, ignores it. The status is that of the other images. fp_workspace_info reports the same.
- Why not "not converged": FP_ENOTCONV means "stopped at max_iter with the last iterate written". A NaN image was never iterated, and more iterations cannot fix it. A caller that retries with a larger max_iter until convergence would loop forever. The NaN gap is the signal, consistent with "NaN in the input gives NaN in the affected output" and with every non-iterative operator, whose status a NaN never changes.
- Code:
- in the NaN branches, set c.gap = std::numeric_limits<double>::quiet_NaN(); c.iters = 0; c.converged = true;: tv_aniso.cpp:264, tv_iso.cpp:305, and edge_tv::Base::read_input (edge_tv.hpp:124), which covers graph TV and tree TV;
- tree TV must not overwrite c.iters = 1 after read_input reports done; check tree_tv.cpp:261-262 and where iters is set;
- image_prox_driver needs no change: gaps[r] (·σ² after the conjugate) becomes NaN, the sum propagates it, and max over ints ignores the 0.
- Device path (specified in P8, stated here so both sides match): test each image for NaN before iterating; such an image gets gap NaN, 0 iterations, and counts as finished, with output and dual NaN.
- Contract text:
- fastprox.h, fp_op_tv_aniso (after "Iterative: fp_info reports the duality gap..."), referenced from fp_op_tv_iso: "An image containing a NaN is not solved: its output and dual are NaN, it adds NaN to fp_info.gap and no iterations, and it does not by itself make the call FP_ENOTCONV."
- fastprox.h, fp_op_graph_tv: the same sentence. Tree TV refers to graph TV's.
- docs/api.md, "Special values" bullet of the image operators (line 266): "A NaN anywhere in an image makes that whole image NaN, and its dual; the image reports a NaN gap and 0 iterations and does not count as unconverged (the rest of the batch is unaffected; fp_info.gap, a sum, is NaN)." The same in the graph TV section (line 422).
- Python IterInfo docstring (bindings/python/fastprox/init.py:76-81): "gap is NaN when an image contains a NaN; converged says whether no image stopped at max_iter (an image with a NaN is not iterated and counts as finished)."
A8.2 — dual round trip.
- dual_impl, to_ws path: in dual_copy<T>(..., to_ws = true) (api.cpp:813), store std::isnan(v) ? 0.0 : v. In the check at api.cpp:835, reject only ±Inf (std::isinf). The seed then holds exactly what the warm start would have used, independent of each kernel's feasibility pass. Graph and tree TV ignore warm starts anyway.
- Kernels keep writing NaN duals. The output, the dual and the gap all signal the NaN image, and branch_image and kink_image keep reading the dual unchanged.
- Contract text:
- fastprox.h:346-347: "set seeds it (FP_EINVAL if an entry is ±Inf; NaN entries, which get returns for an image that held a NaN, are stored as 0)".
- docs/api.md:742: "set rejects ±Inf and stores NaN entries as 0, so a dual read by get can always be set back."
Alternatives considered.
- Gap NaN with converged = false (FP_ENOTCONV): rejected, for the retry loop above. A NaN image would also turn a whole batch into FP_ENOTCONV while every real image converged.
- Keeping gap 0: a false certificate (‖z − z*‖² ≤ 2·gap = 0 for z = NaN).
- The kernels write a zero dual for NaN images: it hides the NaN from get and from the dual-based region logic of branch_image, which could then rebuild finite derivatives for a NaN image.
Constraints. - Determinism: unchanged. A NaN image is detected by a full pass over its entries, whatever the thread count. - Allocation, ABI: not affected. - Spec departure: no; spec Section 10 says "NaN in the input gives NaN in the affected output only, never a hang", and the gap is such an output.
Tests.
- tests/c/test_core.cpp, in test_tv_aniso, test_tv_iso, test_graph_tv and test_tree_tv, with the setup of the first reproducer:
- one NaN image → FP_OK, std::isnan(info.gap), and info.iters equal to that of the clean image alone;
- both images NaN → FP_OK, gap NaN, iters == 0;
- opt.max_iter = 1 with a clean image that needs more → FP_ENOTCONV with a finite gap, so the NaN rule does not mask a real non-convergence;
- fp_workspace_info agrees with the fp_info of each call.
- Dual round trip (test_tv_aniso, test_tv_iso, test_graph_tv): batch of two images with a NaN in image 1, prox with set_warm(ws, 1), then:
- get_dual → FP_OK, with image 1's blocks all NaN and image 0's finite;
- set_dual with that array → FP_OK;
- a prox on clean x is bitwise equal to the same prox after set_dual with the NaN entries replaced by 0;
- a dual with one +Inf → FP_EINVAL.
- Python (tests/python/test_tv_iso.py, test_tv_spatial.py, test_graph_tv.py): IterInfo.gap is NaN and converged is True for a batch with a NaN image. The device-path tests (P8) assert the same.
As implemented.
- C path: as designed. In aniso and iso TV, the NaN gap is set after the singleton structure is saved, so the saved point, which records the gap, is unchanged.
- Tree TV: it sets iters = 1 before read_input, which now resets it to 0, so nothing overwrites it.
- Device path: left to P8, as specified there. Until P8 lands, the device and C certificates still disagree on NaN images.
- Tests: the new C tests fail on the old code for all four operators (the NaN gap) and for the three with warm starts (the set-back). The Python tests fail on the old code too.
- Changed test: test_tv_aniso.py::test_infeasible_warm_start asserted that a NaN seed is rejected, the old contract. It now checks that such a seed gives a certified result, and that an Inf seed is rejected.
Acceptance. The tests pass. Re-running the first reproducer gives gap NaN and iters 0 for all four operators.
A9. A failed fp_workspace_set_branch treats the earlier override differently by operator kind¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. For elementwise operators, set_branch_impl copies side into ws->branch, which is the live override, while it validates. On a value outside −1..1 it sets branch_on = false (api.cpp:780-783), so a valid earlier override is silently dropped. The row and image path (set_structure_branch, api.cpp:719-753) validates into a staging copy and returns FP_EPARAM with the earlier override still applied. There is a second, smaller inconsistency in the row path of apply_structure (api.cpp:696-700): when an override is active, it first restores code from code_default and only then allocates xb, zb and scr. A bad_alloc there leaves override_on true over the default codes.
Design.
- Rule: a failed call leaves the workspace unchanged.
- Elementwise path: validate side in a first pass with the same for_runs loop, reading only, and return FP_EPARAM before any write. Copy in a second pass. Both passes read an int8 array once, which is negligible next to the jvp.
- Row and image path: validate first in the same way (today it writes the staging buffer while validating, which is harmless but uneven).
- apply_structure, row path: allocate xb, zb and scr before the memcpy at api.cpp:696-697.
- Contract text (fastprox.h, after "...or fp_workspace_set_branch(ws, NULL, 0)."): "On failure (FP_EPARAM for a value outside -1..1 or a bad tol, FP_ESHAPE, FP_EINVAL) the workspace is unchanged: an override set earlier still applies." The same sentence goes in docs/api.md under "Derivatives at kinks".
Constraints. - Determinism, allocation, ABI: not affected. - Spec departure: no.
Tests. tests/c/test_kinks.py in Python, or test_kinks() in tests/c/test_core.cpp, for soft (elementwise), tv1d (row), tv_aniso (image) and graph_tv:
- kept prox at a point with kinks, set_branch with +1 at the kink entries, then jvp → A;
- set_branch with one entry 2 → FP_EPARAM, then jvp → bitwise A.
The elementwise case fails today.
As implemented.
- The elementwise path validates in a read-only pass, then copies.
- In the row path of apply_structure, the buffers are allocated before the codes change.
- The image and row validation in set_structure_branch is unchanged. It writes only the staging buffer and returns before applying anything, so a failed call already left the workspace unchanged.
Acceptance. The tests pass.
A10. Derivative calls write the "const" workspace: one call per workspace at a time¶
- Kind: docs · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. fp_jvp, fp_vjp, fp_vjp_device and fp_spectral_jvp take const fp_workspace *, but they write into it:
- ws->dparams (api.cpp:420, 479);
- ws->tot and ws->acc (api.cpp:511; driver.hpp:358, 721);
- the per-thread row and image scratch (driver.hpp:304, 397, 673, 732);
- deriv_res, deriv_its, deriv_residual and deriv_iters (driver.hpp:707);
- ws->spectral, resized on the first call (api.cpp:490).
So two derivative calls on one workspace race. ThreadSanitizer confirms it in gather (driver.hpp:50). The header's "one per concurrent caller" (fastprox.h:83) covers it, but the const invites concurrent backward passes on a shared workspace.
Design.
- fastprox.h:83: typedef struct fp_workspace fp_workspace; /* opaque; calls on one workspace must not overlap */.
- fastprox.h, at the end of the Workspaces block (line 337): "No two calls on one workspace may run at the same time, whatever its constness: fp_jvp, fp_vjp, fp_vjp_device and fp_spectral_jvp take it const because they keep its saved point, but they use its scratch and write their results' bookkeeping into it. Give each thread its own workspace; an fp_op can be shared by any number of threads."
- docs/api.md, "### Threads and determinism": the same paragraph.
- spec.md:512: replace "it holds one scratch region per OpenMP thread" with "it holds one scratch region per pool thread, and no two calls on one workspace may overlap, derivative calls included". The pool replaced OpenMP on 2026-10-15.
- Alternative considered: drop the const. That is an API change; casting away const is legal C, but the change would break C++ callers that hold const pointers, and the freeze is close. A thread-safe derivative path (per-call scratch) would break the no-allocation rule.
Constraints. - Behaviour, allocation, ABI: no change. - Spec departure: no (wording fix of stale text).
Tests. None new. test_concurrent_callers() keeps covering one workspace per thread under the tsan CI job.
As implemented. As designed. The workspace block of the header holds the paragraph next to A6's sentence and A4's saved-point rule.
Acceptance. The four texts agree.
A11. Versioning: "since 1.1" notes on a tree versioned 1.0.0rc1¶
- Kind: docs · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none (but decide before tagging 1.0.0)
Problem. master is versioned 1.0.0rc1:
- CMakeLists.txt:2: project(fastprox VERSION 1.0.0 ...);
- CMakeLists.txt:5: FASTPROX_VERSION_SUFFIX "rc1";
- fastprox.h:22-25: FP_VERSION_* 1.0.0, FP_ABI_VERSION 1.
The v1.0.0rc1 tag (2026-10-20) has none of the spectral operators, fp_prox_steps or the device API. master has 26 commits since that tag, and they document those features as "since 1.1":
- fastprox.h:48, 71, 442, 487, 520;
- docs/api.md:703, 771, 813, 873, 1048, 1116;
- 23 occurrences in all across docs/, bindings/python/fastprox/ and src/core/.
The CHANGELOG lists them under [Unreleased]. ROADMAP.md:143 plans "1.0.0 after some use of the rc: the version without the suffix", from master. Tagging master as 1.0.0 would ship features labelled for a later version and freeze them into the 1.x ABI.
Design. There are two options.
Option 1 (recommended): master ships as 1.0.0, through a second release candidate.
- Version. CMakeLists.txt:5 FASTPROX_VERSION_SUFFIX "rc2"; the version stays 1.0.0 and FP_ABI_VERSION stays 1.
- Remove every "since 1.1" marker. These functions are part of the 1.0 baseline, which needs no "since". The places:
- fastprox.h:48, 71, 442, 487, 520, and the comments that mention 1.1 in src/core/op.hpp, workspace.hpp, driver.hpp and spectral.hpp;
- docs/api.md (the 11 places listed by grep -n "since 1.1" docs/api.md) and docs/spec.md;
- bindings/python/fastprox/init.py, torch.py, jax.py and _interop.py (docstrings).
- CHANGELOG.md: rename ## [Unreleased] to ## [1.0.0rc2] — <date> and open an empty [Unreleased] above it. The audit fixes A1–A10 go in the same section.
- ROADMAP.md:
- in "1.0 — stable", replace the 1.0.0 line (143) with "1.0.0rc2 from master with the spectral operators, fp_prox_steps and the device API; 1.0.0 after some use of it";
- in "1.x — GPU backend" (line 147 onwards), mark the landed device-API item as shipped in 1.0, leaving the kernel stages in 1.x.
- Release. A TestPyPI dry run, then the v1.0.0rc2 tag. The abi job still compares against 0.5.0 (rc tags excluded).
- Decisions row: | 2026-10-22 | The spectral operators, fp_prox_steps and the device API (FP_DEVICE_CUDA, FP_EDEVICE, fp_device_available, fp_op_device_check, the stream setters, fp_workspace_info, fp_vjp_device), planned and documented as 1.1, ship in 1.0.0 after a second release candidate; device arrays still return FP_EINVAL until the 1.x kernels | CMakeLists.txt, fastprox.h, docs/api.md, CHANGELOG.md, ROADMAP.md |
Option 2: 1.0.0 from the rc1 code, master becomes 1.1.
- Release branch: release/1.0.x from v1.0.0rc1, as was done for 0.5.1. Cherry-pick the gating fixes, set the suffix to "", tag v1.0.0.
- master:
- CMakeLists.txt:2 VERSION 1.1.0 with suffix "dev0" (or "a1");
- fastprox.h:23 FP_VERSION_MINOR 1;
- CHANGELOG [Unreleased] stays as the 1.1 section;
- the abi job's baseline becomes v1.0.0 (additions-only --no-added-syms passes, since 1.1 only adds).
- Cost: A1–A3 and A5 touch prox_impl and the drivers, which changed after rc1 (Steps, fp_prox_steps). They would be implemented twice or cherry-picked with conflicts.
Recommendation: Option 1.
- It keeps one line of development, and the 1.0.0-gating fixes in this file apply once.
- It gives the new public surface the release-candidate exposure ROADMAP.md:143 asks for before the freeze.
- The device functions are generic: the stream, the info and fp_vjp_device take no CUDA types, and the plugin table is private and versioned. Freezing them now costs little.
Choose Option 2 only if the maintainer wants the device API to stay out of the frozen ABI until the CUDA kernels exist.
Constraints. - ABI: Option 1 freezes 7 more functions and 2 enum values into ABI 1; Option 2 adds them in 1.1 as additions. - Determinism, allocation: not affected. - Spec departure: Option 1 yes (spec Section 9 places the device values in 1.x). The decisions row is above.
Tests.
- grep -rn "since 1\.1" include docs bindings src returns nothing (Option 1), or every hit is a 1.1 feature (Option 2).
- The abi CI job passes.
- fp_version_string() returns "1.0.0rc2" (Option 1) or "1.1.0dev0" (Option 2) in tests/c/test_header.c.
As implemented. Option 1, as decided on 2026-10-22, with these changes:
- Markers. The 34 "since 1.1" markers became "since 1.0" rather than being removed. Every other addition since 0.5 is already marked "since 1.0", and readers coming from 0.5 need to know which features are new. The markers are in the header, the internal comments, docs/api.md, the docstrings and the tests.
- Version. The suffix is "rc2", and the conda recipe's version matches.
- Version check. test_header checks fp_version_string() against FASTPROX_VERSION_FULL, passed in by CMake, and against the header's macros, so the check needs no edit at a release. It was checked to fail when the expected string differs.
- README. The Install section gained the U7 install paths: a pre-release, a tag from GitHub, and a commit of master, never a working tree.
- CHANGELOG. [Unreleased] became [1.0.0rc2] — 2026-10-23.
- ROADMAP. The rc2 item sits under "1.0 — stable"; the device API is marked as shipping in 1.0, and its kernel stages stay in 1.x.
- Release. The TestPyPI dry run and the tag follow the merge.
Acceptance. The version macros, the CMake version, the CHANGELOG section and every "since" marker agree. The TestPyPI dry run installs and runs the examples.
A12. Enum arguments and fields outside their enumeration¶
- Kind: safety · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Added on 2026-10-22, after the T1 work: the sanitizer run found it, and it is not one of the original findings.
Problem. C enums are ints, so a C caller can pass any int in an enum
argument or field. The C++ library compared those values directly, and in
C++ loading an enum object outside its enumeration's range is undefined.
test_core already passes (fp_device)10 (DLPack's ROCm, which the
header reserves). UBSan reported it at api.cpp:1381, but its reports
recover by default, so the sanitizer job passed. The new C test of this
item triggers 14 reports on the old code. They cover the dtype and
device fields of fp_array, the fp_tv1d_method and fp_spectral_kind
arguments, and the fp_device arguments of fp_device_available and
fp_op_device_check. fp_options.tie is already an int.
Design.
- Add enum_bits(e) in api.cpp. It copies the enum object's bytes into an
int with memcpy, so no enum load happens, and it statically asserts that
the enum is int-sized.
- The boundary validates that int, then converts it: to_arr, fp_op_tv1d,
the spectral kind (taken by reference down to spectral_bit), and the
two device functions. The internal code sees only valid values.
- The sanitizer build adds -fno-sanitize-recover=undefined, so a UBSan
report fails ctest.
Constraints. No change to results, ABI or allocation. Invalid values
return the codes they returned before: FP_EDTYPE for a dtype,
FP_EINVAL for a device or spectral kind, FP_EPARAM for a 1D TV method,
and 0 from fp_device_available.
Tests. tests/c/test_header.c, compiled as C, passes dtype 7, device 10
(and −1 to fp_device_available), spectral kind 5 and 1D TV method 7, and
checks the status codes. Under the sanitizer build, every ctest test runs
with UBSan fatal.
Acceptance. ctest passes in the sanitizer build with UBSan fatal.
C. Execution core¶
Scope. These specs cover the batching, layout and thread-pool code:
- src/core/driver.hpp, layout.hpp, op.hpp (the ElemOp and RowOp helpers), workspace.hpp, spectral.hpp and parallel.cpp;
- the parts of src/api/api.cpp that feed them.
Operator kernels are covered only where a core change needs a hook from them.
Notation.
- Machine. All numbers are from an Apple M5 Pro (5 + 10 cores, fp_default_threads() = 15), Release build (-O3, Apple clang 17), f64, run through the C API with a workspace unless stated otherwise. The load average from other processes was 2–5 during the runs.
- Timing. "Best of 7" is the minimum over 7 timed repetitions. "Median" is over 200 calls.
- Warm and cold. "Warm" means back-to-back calls, so the pool's workers are still spinning. "Cold" means each call follows 300 µs of sleep on the calling thread, so the workers have gone to sleep.
- Threads. "t = k" is fp_options.threads = k. "t = 15" is the default.
- Prototypes. A prototype is a patch to a copy of the repository that was measured and passed test_core (318 241 checks, 0 failures). Where stated, it also passed under ThreadSanitizer.
- Benchmarks. profile_kernels core <case> refers to the benchmark section specified in C11. Each speed acceptance criterion names one of its cases, or a line of uv run python bench/bench_python.py.
C1. Reject overlapping outputs other than exact aliasing; reject stride-0 outputs¶
- Kind: safety · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: none
Problem. The drivers recognise aliasing only by pointer equality: driver.hpp:151 (rows), :583 (images), :301 and :394 (jvp/vjp). The elementwise path (driver.hpp:123-146) has no check at all.
- In-place transpose: soft thresholding (λ = 0.5) on a 512 × 512 C-order array, with
outthe same buffer but strides {1, 512}, returnsFP_OKwith 96 683 of 262 144 entries wrong (t = 1 and t = 8 alike). - Shifted rows: tv1d on 4096 × 64, with
out.data = x.data − 64(out is x shifted up by one row), gives 0 wrong entries at t = 1 and up to 960 wrong at t = 8 over 5 runs. The result then depends on the thread count, which breaks the determinism guarantee, and it is a data race. - Stride-0 output: soft on 2^20 entries into an
outof stride 0 at t = 8 returnsFP_OK. It gave 3 distinct results in 20 runs, and ThreadSanitizer reports a race inElemOp::run_prox(op.hpp:391).
fastprox.h:12 says only "out may alias the input"; docs/api.md:652 calls other overlaps undefined.
Design. The decision is already taken; this is the design within it.
- New helpers in
src/core/layout.hpp:
// Exact aliasing: same data pointer and the same stride on every dimension
// of extent > 1 (strides of extent-1 dimensions are never used).
inline bool same_layout(const Arr &a, const Arr &b) noexcept {
if (a.data != b.data || a.ndim != b.ndim) return false;
for (int d = 0; d < a.ndim; ++d)
if (a.shape[d] != b.shape[d] || (a.shape[d] > 1 && a.strides[d] != b.strides[d])) return false;
return true;
}
// Byte range [lo, hi] of the entries of a, numel > 0. False when an
// offset overflows int64 (the caller returns FP_ESHAPE).
inline bool byte_range(const Arr &a, uintptr_t &lo, uintptr_t &hi) noexcept {
int64_t mn = 0, mx = 0;
for (int d = 0; d < a.ndim; ++d) {
int64_t e;
if (__builtin_mul_overflow(a.shape[d] - 1, a.strides[d], &e)) return false; // _mul128 on MSVC
if (e < 0 ? __builtin_add_overflow(mn, e, &mn) : __builtin_add_overflow(mx, e, &mx)) return false;
}
const int64_t item = static_cast<int64_t>(a.itemsize());
int64_t blo, bhi;
if (__builtin_mul_overflow(mn, item, &blo) || __builtin_mul_overflow(mx, item, &bhi)) return false;
lo = reinterpret_cast<uintptr_t>(a.data) + static_cast<uintptr_t>(blo); // modular: blo <= 0
hi = reinterpret_cast<uintptr_t>(a.data) + static_cast<uintptr_t>(bhi) + static_cast<uintptr_t>(item - 1);
return true;
}
// An output array may be written: no stride 0 on a dimension of extent > 1,
// and either exactly the input or disjoint from its byte range.
// in may be null (no input to compare).
inline fp_status check_output(const Arr *in, const Arr &out) noexcept {
if (out.numel() == 0) return FP_OK;
for (int d = 0; d < out.ndim; ++d)
if (out.shape[d] > 1 && out.strides[d] == 0) return FP_EINVAL;
if (!in || in->numel() == 0 || same_layout(*in, out)) return FP_OK;
uintptr_t a0, a1, b0, b1;
if (!byte_range(*in, a0, a1) || !byte_range(out, b0, b1)) return FP_ESHAPE;
return (a0 <= b1 && b0 <= a1) ? FP_EINVAL : FP_OK;
}
A shape product that overflows is A2. byte_range only refuses an extent it cannot represent.
- Call sites in
src/api/api.cpp. Each check runs afterto_arrand the shape and dtype checks, and before any state change: prox_impl:check_output(&x, out). Withfp_prox_steps, alsocheck_output(&steps_arr, out), since the steps are read during the work (Steps::at). Both run before theif (ws) { ws->valid = false; ... }block, so a rejected call leaves the saved structure of the previous call usable.jvp_impl:check_output(dxa ? &dx : nullptr, out).dparamsis copied before the work, so it is not compared.vjp_impl:check_output(&g, gx).gparamsandggammaare written after the work, fromws->tot, so they are not compared.-
Same helper, stride-0 rule only (
in= null), for the other array outputs:fp_value'sout,fp_workspace_kinks'smaskandfp_workspace_get_dual'sout. The spectral functions already document "b and r must not overlap a, s or z"; enforce it withcheck_output(&a, b),check_output(&s, b),check_output(&z, b), and the same forr. -
Drivers. No change to the algorithms.
alias(x.data == out.data) now impliessame_layout, so the existing staging (alias && distinct,x.data == out.datain the image driver) stays correct. Add a comment saying so.
Contract text.
fastprox.h, file header (line 12), replace "and out may alias the input." with:
and
outmay be the input array itself (the same data pointer, shape and strides); an output that otherwise shares a byte with an input, or that has stride 0 on a dimension of extent > 1, is FP_EINVAL.
fastprox.h, the "Entry points" comment (around line 436), replace "out may alias x." with:
out may be x itself (same data, shape and strides). Otherwise out must not intersect the byte range [lowest, highest byte] of x (or of the steps array, for fp_prox_steps): FP_EINVAL, checked before any work, so the workspace keeps the structure of its previous call. The test is conservative: two disjoint interleaved views of one buffer (x = a[0::2], out = a[1::2]) are rejected. A dimension of extent > 1 with stride 0 in out is FP_EINVAL. The same rules hold for dx and dout in fp_jvp and for g and gx in fp_vjp.
docs/api.md:652, replace the sentence with:
The library never frees
data.outmay be the input array itself: the same data pointer, shape and strides, as infp_prox(op, g, &x, &x, ...). Any other overlap between the byte ranges of an input and an output returnsFP_EINVALbefore any work, including views that interleave without sharing an element, such asa[0::2]anda[1::2]. An output dimension of extent > 1 with stride 0 also returnsFP_EINVAL. The same rules apply todxanddoutinfp_jvpand togandgxinfp_vjp. The Python package copies its input when it may share memory withoutin another layout, so Python callers never see this error.
Alternatives considered.
- An exact element-overlap test: an integer-programming problem in general (NumPy's may_share_memory has a work bound). Rejected for its cost and complexity.
- Staging through a copy instead of rejecting: it needs an O(numel) allocation in the entry points, which breaks the no-allocation rule.
- A general internal-overlap test for out (sort dims by |stride| and require each stride to exceed the span of the smaller ones): not in the decision. It would catch as_strided outputs. Deferred.
Constraints.
- Determinism: this restores it for every accepted call, since a rejected overlap was the only thread-dependent path found.
- Allocation: none; the check is O(ndim).
- ABI: no signature change. Calls that were undefined now return FP_EINVAL.
- Clean room: n/a.
- Departure from docs/spec.md (Section 4, "out may alias x"). Decisions row:
| 2026-10-22 | out may alias x only exactly (same data, shape and strides); any other byte-range overlap of an output with an input, or a stride-0 output dimension of extent > 1, is FP_EINVAL, checked before any work (conservative: disjoint interleaved views are rejected). Same rule for dx/dout and g/gx; the Python package copies x instead | fastprox.h, src/core/layout.hpp, src/api/api.cpp, docs/api.md |
Tests. A new test_aliasing() in tests/c/test_core.cpp:
- Exact alias accepted. For soft, tv1d and tv_aniso 2D, at t ∈ {1, 8}: out == x, both C-contiguous and as a view with negative, non-unit strides (shape (300, 257) read as x[i, j] = buf[(299 − i)·514 + (256 − j)·2], i.e. strides {−514, −2}). Assert FP_OK and equality, bit for bit, with the out-of-place result.
- Overlaps rejected, before any work. Each case must return FP_EINVAL and leave a sentinel-filled out untouched:
- the in-place transpose (512², strides {1, 512} against {512, 1});
- tv1d on 4096 × 64 with out.data = x.data ± 64;
- x = a[0::2], out = a[1::2];
- out = x reversed (same buffer, negated strides).
- Stride-0 outputs. out with stride 0 on a dimension of extent 1000 → FP_EINVAL. Stride 0 on a dimension of extent 1 → FP_OK.
- Workspace survives a rejected call. Run a kept prox, then a rejected prox (overlap) on the same workspace, then fp_jvp. The jvp must be FP_OK and equal to the jvp before the rejected call.
- Derivatives. fp_jvp with dx and dout partially overlapping → FP_EINVAL; with dx == dout exactly → FP_OK, equal to the out-of-place result. The same pair of checks for fp_vjp with g and gx.
- Steps. fp_prox_steps with the steps array inside out's byte range → FP_EINVAL.
- Python. The copy itself is P2. Add one assertion to its tests that prox(x, out=x.T) equals prox(x).T.
As implemented (with A2 and P2 in one change):
- byte_range needs no overflow checks: A2 bounds every descriptor's entry count and offset span by 2⁶⁰ − 1 in to_arr.
- fp_value's output gets the full overlap check against x, not only the stride rule. It writes a row's value after reading that row, so an overlapping output corrupts later rows, as in the prox.
- P2 does not copy up front. A may_share_memory test on every call cost 0.15–0.4 µs, against a 0.05 µs budget. Instead the Python layer calls the C library as is, and after a rejection diagnoses the cause:
- an out that is read-only or has a zero stride raises a clear ValueError;
- inputs that share memory with out are copied and the call retried;
- anything else re-raises the original error.
The C check runs before any work, so a retry is safe, and the common path costs nothing (0.79 against 0.76–0.84 µs for fp.soft_threshold on 100 entries).
Acceptance. test_aliasing passes in the c, sanitizers and tsan CI jobs. The overhead line of uv run python bench/bench_python.py ("Per-call time from Python, n = 100") moves by less than 0.05 µs.
C2. Size per-thread scratch for the threads a row operator can use¶
- Kind: scaling · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: none
Problem. build_workspace sizes the row scratch as thread_bytes × nthreads (api.cpp:171,215), where thread_bytes = round64(4 n item) + round64(scratch_bytes(n)) (driver.hpp:30). That is true even when the batch has fewer rows than threads, although a row never uses more than one thread.
Measured for tv1d, one row of length n, as the bytes fp_workspace_create allocates:
| n | t = 1 | t = 15 |
|---|---|---|
| 2^16 | 5.5 MB | 82.5 MB |
| 2^20 | 88 MB | 1320 MB |
| 2^23 | 704 MB | 10 560 MB |
Every fp_prox call without a workspace allocates the same. Pages are touched lazily, so the time is unchanged. But on Linux with heuristic overcommit, or on Windows with commit accounting, a 10 GB request for one 8M-sample signal can return FP_ENOMEM.
Design. In build_workspace (api.cpp), for operators that are neither elementwise nor image operators, just before w.thread_bytes is computed:
// A row runs on one thread: no more scratch regions than rows.
if (!elem) w.nthreads = static_cast<int>(std::min<int64_t>(w.nthreads, std::max<int64_t>(w.rows, 1)));
Every row driver already clamps to it: prox_impl does nt = std::min(nt, ws->nthreads), and jvp_driver/vjp_driver start from ws.nthreads. And parallel_run never hands a tid ≥ the number of pieces ≤ rows. So no driver change is needed.
For elementwise operators w.nthreads stays as requested, since their chunks split one row. Their only per-thread scratch is the spectral one. Change spectral_jvp_impl (api.cpp:489) to need = spectral_doubles(n) * min(nthreads, max(rows, 1)), and spectral_driver (spectral.hpp:80) to nt = std::min<int64_t>(ws.nthreads, rows).
Image operators are left as they are: image_sets = min(B, nthreads) already, and inside one image the threads are used. Measured for one 2D image, t = 15 against t = 1: image_thread adds 0.2 MB (64²) to 13.6 MB (4096²), against a per-image working set of 34 MB (512²) and 2177 MB (4096²). The working set itself, about 17 × the image for 2D TV, belongs to the kernels and is not specified in this audit (a candidate for a later pass).
With this, row scratch is at most rows × thread_bytes ≈ (4·item + scratch_bytes(n)/n) · numel, a constant times the data. For tv1d the kernel scratch is 56 B per sample.
Alternative rejected: a separate scratch_threads field. It is the same number, and every row driver already clamps to nthreads.
Constraints.
- Determinism: unchanged. The thread count never changes results, and rows ≤ nthreads.
- Allocation: fewer bytes. Still everything is sized in build_workspace.
- ABI: none.
- Spec: no departure.
Tests. In tests/c/test_core.cpp:
- Extend the operator new counter with a byte total (g_alloc_bytes).
- A tv1d workspace for shape (1, 2^16) at t = 0 must allocate at most 1.2 × the bytes of the t = 1 workspace.
- The same for shape (3, 2^16), with a bound of 3.6 ×.
- The existing test_no_allocation passes unchanged.
As implemented.
- As designed, in build_workspace, spectral_jvp_impl and spectral_driver.
- The test counts the bytes operator new receives, for shapes (1, 2¹⁶) and (3, 2¹⁶) at the default thread count and at 64 threads. Both are within 1.2× the rows times the one-thread size; the old code fails both.
Acceptance. profile_kernels core wsbytes (C11), at t = 0, prints a workspace for one tv1d row of 2^20 that drops from 1320 MB to 88 MB, and one of 2^23 from 10 560 MB to 704 MB. The prototype measured exactly these. The prox time it prints for the same rows stays within ±3%.
C3. Cap fp_options.threads¶
- Kind: safety · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. resolve_threads (api.cpp:237) passes any positive o.threads through. Only the environment variables are capped, at 4096 (parallel.cpp:59).
Soft on 2^26 entries (4096 chunks) with threads = 100000 created 4095 permanent pool threads. The call took 459 ms against 33 ms at t = 1, and the threads stay for the life of the process. A larger array, or a row operator with many rows, would create up to 0xffff workers.
Design. The cap is max_threads() = clamp(4 × available_cpus(), 1, 4096).
- Export it from the pool: in
parallel.hpp,int max_threads() noexcept;. Inparallel.cppit returns the cap, computed once withdefault_threads()'s method (static local). TheFASTPROX_NO_THREADSbuild returns 1. resolve_threadsbecomesconst int t = o.threads > 0 ? o.threads : default_threads(); return std::min(t, max_threads());.- The environment default is also clamped:
default_threads()returnsmin(env value, max_threads()). This keepsFASTPROX_NUM_THREADS=10000on a 16-CPU machine from creating 4095 threads. - Negative
o.threadskeeps its current meaning (the default). It is not rejected: that would change behaviour callers may rely on.
Why 4 × CPUs: the pool runs one job at a time, so more threads than CPUs gain nothing. A margin of 4 keeps containers whose affinity mask under-reports the CPUs from being throttled. 4096 matches the existing environment cap.
Contract text, fastprox.h:87:
int threads; /* 0 = fp_default_threads(), 1 = serial; values above 4 x the available CPUs (at most 4096) count as that cap */
Contract text, docs/api.md "Threads and determinism", after the first sentence:
Thread counts above four times the CPUs the process may run on (and above 4096) are reduced to that cap, for
opt.threadsand for the environment variables alike.
Alternatives rejected:
- Capping at the CPU count: it is unreliable under cgroups, and the margin is harmless.
- Returning FP_EINVAL: a large value is a reasonable "use everything".
Constraints.
- Determinism: unchanged.
- Allocation: fewer bytes for absurd requests.
- ABI: none.
- Spec: no departure. Decisions row: | 2026-10-22 | fp_options.threads and the thread environment variables are capped at min(4096, 4 x the available CPUs) | src/core/parallel.cpp, src/api/api.cpp, fastprox.h |
Tests. In tests/c/test_core.cpp, fp_prox (soft) with threads = 100000 on 2^26 entries returns FP_OK, and afterwards the process holds at most 4 × hardware_concurrency + 2 threads. On macOS count them with task_threads; on Linux count /proc/self/task; skip elsewhere. Run this test last, since it grows the pool.
As implemented.
- max_threads() is in the pool.
- resolve_threads, default_threads() and parallel_run itself all apply the cap, so internal callers are covered as well.
- The test uses 2²² entries (256 chunks) instead of 2²⁶, which reaches the cap with 64 MB instead of 1 GB of data.
Acceptance. The test passes; on the M5 Pro the thread count after the call is at most 61, instead of 4096.
C4. Vectorise the elementwise prox loops, contiguous and in place¶
- Kind: speed · Severity: high · Target: 1.0.x · Effort: M · Depends on: C11
Problem. ElemOp::run_prox (op.hpp:352-393) calls D::prox1 per entry. prox1 reads the operator's members (lambda, w.size(), w[i]), and since z is a T* that may alias them, clang reloads them in every iteration. The disassembly shows the explicit contiguous branch (op.hpp:385) compiled to a scalar loop for soft thresholding, with the lambda load and the w.size() select inside it. Only the generic strided loop got a runtime-versioned vector body.
In place (x == z), every operator falls back to scalar, because the vectoriser's overlap check fails when the ranges coincide.
Out of place, serial, soft takes 0.48 ns per entry against 0.155 ns for a hand-written loop of the same formula. In place, on 2^16 entries at t = 1:
| Operator | In place, ns per entry |
|---|---|
| soft | 0.47 |
| elastic net | 0.44 |
| huber | 0.56 |
| box | 0.38 |
| nonneg | 0.25 |
| mcp | 0.63 |
| l0 | 0.76 |
Out of place, elastic net and huber are vectorised (0.20) but box, mcp and l0 are not (0.40–0.56).
Design.
- A by-value kernel per operator.
ElemOp<D>gains an optional CRTP hook.Dmay define:
static constexpr bool kUniformKernel = true;
template <class T> struct UK { /* per-call constants, by value */ T operator()(T x) const noexcept; };
template <class T> UK<T> uniform_kernel(double gamma) const noexcept; // when !axis_params()
static constexpr bool kIndexedKernel = true;
template <class T> struct IK { /* constants + const double *w (weights pointer) */ T operator()(T x, int64_t i) const noexcept; };
template <class T> IK<T> indexed_kernel(double gamma) const noexcept; // when axis_params()
ElemOp defaults both flags to false. The kernel struct must hold only values and const double * arrays copied out of this: no reference to the operator. It must compute exactly the expression prox1 computes, so that results are bit-identical. Constants that prox1 forms per entry may be formed once per call only if the expression is the same, as with soft's gamma * lambda * w[0].
Soft's uniform kernel, as prototyped:
template <class T> struct UK {
double tau;
T operator()(T x) const noexcept {
const double xd = static_cast<double>(x);
return static_cast<T>(std::copysign(pos(std::fabs(xd) - tau), xd));
}
};
template <class T> UK<T> uniform_kernel(double gamma) const noexcept { return {gamma * lambda * w[0]}; }
The indexed kernel holds double gl = gamma * lambda; const double *w; and computes tau = gl * w[i]. That equals today's gamma * lambda * w[k] bit for bit only because that expression parses as (gamma * lambda) * w[k]. Keep that parse.
- Loops in
ElemOp, used when the flag is set,conj == false, and the operator has no ties (!D::kTies):
template <class T, class K>
static void kernel_loops(const K k, const T *x, int64_t xs, T *z, int64_t zs, int64_t len, T *save) noexcept {
if (save) { for (i) { const T xi = x[i * xs]; save[i] = xi; z[i * zs] = k(xi); } }
else if (xs == 1 && zs == 1) {
if (static_cast<const void *>(x) == static_cast<const void *>(z)) { T *p = z; for (i) p[i] = k(p[i]); } // in place
else for (i) z[i] = k(x[i]);
} else for (i) z[i * zs] = k(x[i * xs]);
}
The indexed form passes widx + i * wstep as the second argument. Since k is a local copy, the in-place loop reads and writes one pointer at one index, so the compiler needs neither alias nor overlap checks. In run_prox, before the existing code:
if constexpr (D::kUniformKernel) if (!conj && !this->axis_params()) { kernel_loops<T>(d.template uniform_kernel<T>(gamma), ...); return 0; }
if constexpr (D::kIndexedKernel) if (!conj && this->axis_params()) { kernel_loops<T>(d.template indexed_kernel<T>(gamma), widx, wstep, ...); return 0; }
The conjugate loop gets the same treatment in a second step: z = y − σ k(y/σ), same structure.
-
Operators to convert. The elementwise maps without ties: soft, elastic net, huber, box (uniform bounds and per-entry bounds), nonneg, log barrier and ℓ1-box. The nonconvex maps with
kTies(l0, lq, scad, mcp, log penalty) need a tie-counting kernel returning{T z; bool tied}. That is a follow-up; leave them on the existing path. -
Verify with
bench/asm_summary.py: add"run_prox"and"kernel_loops"toKERNELS. The contiguous and in-place loops must show vector instructions and no conditional branch in the loop body.
Alternatives rejected (both prototyped):
- __restrict on z: undefined for x == z.
- #pragma clang loop vectorize(assume_safety): clang-only, and it does not stop member reloads.
- Deleting the contiguous branch: soft went from 0.48 to 0.155, but box went from 0.40 to 0.51 and l0 from 0.56 to 0.81.
- Blocking through a 256-entry stack buffer: in place it gained 1.5–2.3× for most operators, but nonneg out of place regressed from 0.11 to 0.31.
Constraints.
- Determinism: same expressions, same order, so results are bit-identical to today (the prototype passed test_core's reference checks).
- Allocation: none.
- ABI: none, internal.
- Spec: no departure.
Tests.
- In tests/c/test_core.cpp, for every converted operator, f32 and f64, on contiguous, in-place, strided (stride 3), keep and no-keep layouts: assert outputs are bit-identical to prox1 applied per entry, with the scalar reference loop already used by test_batching.
- In tests/python/test_operators.py's stride tests, add a 2^16-entry in-place case for every elementwise entry of CASES: out=x must equal prox(x.copy()) exactly.
As implemented. As designed, for soft, the elastic net, Huber, the box, nonneg, the log barrier and ℓ1-box.
- Indexed kernels. Each parameter is a Par<per_entry> chosen at compile time: a value, or a pointer indexed by the entry. A runtime stride (p[i * s], first try) made the box with per-entry bounds 6× slower (0.31 → 1.84 ns), since the vectoriser judged the strided loads not worth it. with_indexed_kernel dispatches once per run to the combination that applies.
- Bits. The same as master, checked with a program that hashes the outputs of every converted operator, contiguous, in place, strided by 3 and kept at 4 threads, f32 and f64, linked once against master's library and once against this one.
- Weighted soft. The first measurement, 0.69 ns per entry against a target of 0.25, came from elsewhere: sampling showed 72% of the call in Soft::step_scale(), whose vmax(w) scanned the n weights on each prox call for A1's step-range check. Ten operators did the same; the maximum is now a member formed at construction. Weighted soft is at 0.18 ns.
- Measured (profile_kernels core elem, alternating master and this branch, ns per entry):
| Operator | Out of place, before → after | In place, before → after |
|---|---|---|
| soft | 0.41–0.44 → 0.16 | 0.44 → 0.16 |
| weighted soft | 0.92 → 0.18 | 0.88 → 0.18 |
| elastic net | 0.19 → 0.19 | 0.60 → 0.20 |
| huber | 0.22 → 0.22 | 0.63 → 0.22 |
| box | 0.36 → 0.14 | 0.35 → 0.13 |
| nonneg | 0.09 → 0.10 | 0.23 → 0.12 |
Through Python on 10⁶ entries: box with per-entry bounds 0.30 → 0.28 ns, and ℓ1-box 1.26 → 0.87 ns.
- Tests. test_elementwise_fast_paths (C) covers every elementwise operator of derivative_cases: contiguous, in place, strided, f32 in place, and kept along both axes. test_in_place_long_rows (Python) covers every elementwise entry of CASES, in place.
Acceptance. profile_kernels core elem (C11), t = 1, 2^16 entries:
| Case | Before → after (ns per entry) | Status |
|---|---|---|
| soft, out of place | 0.465 → 0.167 | prototyped |
| soft, in place | 0.465 → 0.151 | prototyped |
| other converted maps, out of place and in place | → ≤ 0.25 | target |
| nonneg, out of place | ≤ 0.11 (no regression) | target |
| weighted soft (weights of length n) | 0.47 → ≤ 0.25 with the indexed kernel | target, not prototyped |
The tv1d and simplex lines of uv run python bench/bench_python.py do not regress.
C5. Elementwise jvp and vjp: register accumulators, branch-free pieces, skip unrequested sums¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: M · Depends on: C4 (same hook style), C11
Problem. Elementwise derivatives cost 7–20 × the prox. On 256 × 1000 entries, normal random x and g, kept structure, t = 1, best of 30:
| Operator | prox | jvp | vjp |
|---|---|---|---|
| soft (λ = 0.5) | 45 µs | 704 µs | 697 µs |
| nonneg | 57 µs | 56 µs | 409 µs |
| huber (λ = 0.5, δ = 1) | 45 µs | 371 µs | 540 µs |
Through Python, nonneg (256 × 1000, one thread) takes 34, 62 and 616 µs for prox, jvp and vjp. There are three causes:
- (a) Memory accumulators.
run_vjp_(op.hpp:~455-480) accumulatesacc[e.pi[k]] += e.pv[k] * giandacc[P] += e.dg * giin memory.gxstores may aliasacc, so every entry stores and reloads, a loop-carried chain through memory. Nonneg vjp costs 7 × its prox for anacc[P] += 0 * gichain alone. - (b) Unrequested sums. These sums are computed even when the caller passes
ggamma = NULLandgparams = NULL. - (c) Branchy piece selection.
piece_of/sgnbranch on the data and mispredict. With x = 5 + |N(0, 1)| (a single piece everywhere), soft jvp drops from 704 to 268 µs.
Design.
- Skip the sums when they are not requested.
vjp_implcomputesconst bool want = ggamma != nullptr || (gpa && P > 0);and passes it tovjp_driver<T>(..., tot, want).- The elementwise branch of
vjp_driverpasseswant ? a : nullptrasacctovjp_run. ElemOp::run_vjpdispatchesacc ? run_vjp_<T, Side, true> : run_vjp_<T, Side, false>. WithAcc = false, the loop computes onlygx(and nothing of the conjugate extra term).- Row and image operators keep receiving
acc; their kernels assume it is non-null. Extending the skip to them is a per-kernel follow-up. - The Python layer should pass NULL for
ggammaandgparamswhen no parameter gradient is requested; that is P11, item 7. -
Also skip the gamma slot when the operator's
dz/dγis identically zero: a newstatic constexpr bool kGammaFreeonD(nonneg, box, the projections). This changes one non-finite case: todayggamma = Σ 0·g_iis NaN when someg_iis ±inf, and afterwards it is exactly 0, which is the correct derivative. Document it inCHANGELOG.md. -
Register accumulators with the order unchanged.
- In
run_vjp_<T, Side, true>, keep the running value of each slot in a register for the run:aP = acc[P],aX = acc[P+1]at entry,acc[P] = aP,acc[P+1] = aXat exit. - For parameters, specialise on
NP: when!axis_params()andP ∈ {0, 1, 2}, holdl0 = acc[0]andl1 = acc[1]in registers. - With
NP = 1, every active contribution goes tol0:l0 += (k < e.np) ? e.pv[k] * gi : 0.0fork = 0, 1, 2. - With
NP = 2:l0 += (k < e.np && e.pi[k] == 0) ? c : 0.0, and likewise forl1. - Otherwise (
axis_params(), orP > 2) parameters stay in memory,acc[e.pi[k]] += .... Those addresses vary along the axis, so there is no chain. - Why the result is bit-identical to accumulating in memory in the same order: a slot receives the same sequence of additions in the same order. The running value lives in a register across the entries of one run and is written back at the run's end, and the next run reloads it. The block's runs are visited in C order, so the sequence per slot is the C-order sequence of the block's entries. That is independent of how the plan splits the block into runs (array strides), and of the thread count, since the block boundaries
nch·b/nbandreduction_blocksdo not depend on either. - The
+ 0.0added for inactive slots is the identity: a slot starts at+0.0and a round-to-nearest sum is−0.0only if both terms are, so a slot is never−0.0, andx + 0.0 == xfor every other x, NaN included. The select happens after the multiply, so a NaNg_ion an inactive slot contributes nothing, as today. -
One caveat: the old statement
acc[i] += pv * gimay be contracted to a fused multiply-add (clang-ffp-contract=on, GCC's defaultfast); the select form is not. Results may therefore change in the last bit once relative to 1.0.0rc1: in the prototype, huber's θ₀ gradient changed in its last bit, while soft, nonneg and allgxandggammavalues were identical. They stay bit-identical for any thread count and any layout, which is the guarantee. -
Branch-free piece selection on the hot path.
ElemOpgains two optional hooks, with defaults derived fromderiv1:JDg jdg1(double x, double gamma, int64_t w) const, wherestruct JDg { double j, dg; }: used byrun_jvp_whendparams == nullptrand there is no side override;double j1(double x, double gamma, int64_t w) const: used byrun_vjp_<.., false>.
- They must be written with value selects (
std::fabs(x) > tauturned into a 0/1 double viastatic_cast<double>(cond),std::copysign), not with?:on data-dependent conditions, which clang turned back into branches in the prototype (soft jvp stayed at 694 µs with a?:-basedderiv1). - Soft:
j1 = static_cast<double>(std::fabs(x) > tau),dg = -copysign(1, x) * lambda * w * j1. Nonneg:j1 = static_cast<double>(!(x < 0)). deriv1itself keeps its current form for the parameter path and forderiv_at.- Check with
bench/asm_summary.py(add"jvp_run"and"vjp_run"toKERNELS): no conditional branch inside the per-entry loop.
Alternatives considered.
- Split each slot into 4 lanes by C-order position mod 4, combined ((l0 + l1) + (l2 + l3)) at the block end. It is still deterministic and layout-independent, and would lift the remaining fadd-latency bound (vjp with all sums about 1.1 ns per entry for soft). It is rejected for now because it changes every gradient's bits once more. A candidate for 1.1 if parameter-gradient vjp throughput matters.
- Accumulating into a stack array la[4] indexed by pi: prototyped, and slower (soft vjp 707 → 1134 µs), because of store-to-load chains through the stack.
Constraints.
- Determinism: fixed per-slot C-order sequence within fixed blocks, as argued above. The thread count and the layout never change a bit.
- Allocation: none.
- ABI: none; the NULL arguments already exist.
- Spec: no departure. CHANGELOG notes the last-bit change and the kGammaFree ±inf case.
Tests. In tests/c/test_core.cpp, test_batching already compares batched against per-row. Add:
- vjp with ggamma = NULL and gparams = NULL gives gx bit-identical to the call with both requested, for every elementwise operator in derivative_cases;
- ggamma and gparams are bit-identical across t ∈ {1, 2, 15} and across layouts (C order, negative strides, axis 0), on 256 × 1000;
- nonneg with one g_i = inf: ggamma == 0.
The existing finite-difference and adjointness tests in tests/python/test_operators.py cover correctness.
As implemented. Parts 1 and 2 as designed; part 3 differently, and with two targets left open.
- Part 1. want in vjp_impl and vjp_driver; acc null gives a gx-only loop. kGammaFree is set on the box and nonneg.
- Part 2. dispatch_vjp picks run_vjp_<T, Side, Acc, NP> with NP 0, 1, 2 or −1. With contraction off since #105, the register sums are the same bits as the memory ones: no last-bit change at all, checked against master.
- Part 3. No jdg1/j1 hooks. Instead:
- soft's deriv1 forms both pieces and selects (on ? a : 0.0), which compiles to selects;
- the jvp (no dparams, not conjugate, contiguous) and the gx-only vjp (contiguous, not conjugate) get loops without runtime strides, which the compiler vectorises.
- Huber's branch-free deriv1 was slower (jvp 385 → 412 µs): the division x / c then runs for every entry rather than the two thirds inside. It keeps its branch, and a comment says why.
- Bits. Identical to master for the jvp (with and without dgamma and dparams), the vjp with every sum, and the gx-only vjp: every elementwise operator, the nonconvex ones included, prox and conjugate, at 1 and 4 threads. The one intended change: a projection's gamma gradient with an infinite cotangent (0 now, NaN before).
- Measured (profile_kernels core deriv, 256 × 1000, alternating master and this branch, µs):
| Operator | Measure | Before → after |
|---|---|---|
| soft | jvp, t = 1 / t = 15 | 540–603 → 177–180 / 47–49 → 18 |
| soft | vjp, all sums, t = 1 / t = 15 | 573–664 → 349–360 / 68–96 → 37–38 |
| soft | vjp, gx only, t = 1 / t = 15 | 588–646 → 47–51 / 70–101 → 7–9 |
| nonneg | vjp, all sums, t = 1 | 418–488 → 55–57 |
| huber | jvp, t = 1 | 384–403 → 140–146 |
| huber | vjp, all sums / gx only, t = 1 | 808–852 → 449–505 / 808–851 → 40–48 |
- Open: soft's jvp at ≤ 2 × prox (≤ 100 µs; it is at 177–180), and soft's vjp with all sums at 288 µs (it is at 349–360).
- Why: the remaining cost is the fadd latency of the in-order sums, and in the jvp the
ElemDerivrecord's integer fields. - Next steps would be the spec's four-lane sums (a change of bits, for 1.1) and a jvp hook that returns only j and dg.
- Tests.
test_elementwise_fast_pathscovers: - gx with no sums equal to gx with all sums;
- the sums bit-identical for t ∈ {1, 2, 15}, within each layout (C order, reversed rows, transposed along axis 0);
- a projection's gamma gradient exactly 0 with an infinite cotangent. This is the only one of these checks that fails against master's library.
Acceptance. profile_kernels core deriv (C11): 256 × 1000, normal random x and g, best of 30. The prototype numbers are on that setup.
| Operator | Measure | Before → after | Status |
|---|---|---|---|
| soft | vjp, all sums, t = 1 | 697 → 288 µs | prototyped |
| soft | vjp, all sums, t = 15 | 78 → 33 µs | prototyped |
| soft | vjp, gx only, t = 1 | 707 → 49 µs | prototyped |
| soft | vjp, gx only, t = 15 | 87 → 7.7 µs | prototyped |
| nonneg | vjp, all sums, t = 1 | 409 → 127 µs | prototyped; target ≤ 45 µs with kGammaFree |
| nonneg | vjp, gx only, t = 1 | 400 → 41 µs | prototyped |
| nonneg | jvp, t = 1 | 56 → 39 µs | prototyped |
| huber | vjp, gx only, t = 1 | 564 → 41 µs | prototyped |
| soft, huber | jvp with jdg1, t = 1 |
704 / 371 → ≤ 100 µs (≤ 2 × prox) | target, not prototyped |
The lines of uv run python bench/bench_python.py do not regress.
C6. Pool: close the job when its pieces are done, instead of waiting for every worker¶
- Kind: speed · Severity: high · Target: 1.0.x · Effort: M · Depends on: C11
Problem. Pool::run (parallel.cpp:117-143) sets pending_ = nw and returns only when all nw workers have woken, run work() and decremented it (:183-188), even a worker that wakes after the caller has done all the pieces.
The caller therefore pays the slowest worker's wake-up or preemption latency on every call. An empty 15-thread job (parallel_for(15, 15, 1, f) with a trivial body) shows it:
| Condition | Median | p99 |
|---|---|---|
| warm, light load | 1.6 µs | 75–100 µs |
| warm, load average 4.9 | 82 µs | — |
| cold, nt = 8 | 65 µs | — |
| cold, nt = 15 | 100–107 µs | — |
Soft at t = 15 is 20 µs warm but 98 µs cold at 2^18 entries, and 41 µs warm against 128 µs cold at 2^20. Under background load, soft at 2^18 entries took 85 µs at t = 15 and 20 µs at t = 8.
Replacing the condition variable by std::atomic::wait changed nothing: the cost is OS wake and preemption latency, not mutex contention.
Design. One control word replaces state_ and pending_:
ctl_ (std::atomic<uint64_t>):
bits 63..33 generation (31 bits)
bit 32 CLOSED
bits 31..16 nw: workers allowed into this job (index < nw)
bits 15..0 active: workers inside this job
Job fields fn_, ctx_, count_, grain_ and next_ keep their meaning. They are written only by the caller holding run_m_, and only while no worker is inside a job (the previous job is closed with active == 0).
Caller (run):
1. try_lock(run_m_), grow(nt − 1), nw = min(nt − 1, nthreads_, 0xffff) — unchanged.
2. Write fn_, ctx_, count_, grain_, and next_ = 0 (relaxed).
3. Under m_: gen = gen(ctl_) + 1; ctl_.store(gen << 33 | nw << 16, release). Then wake_.notify_all() if nw > 0.
4. work(0), which returns when next_ ≥ count_: every piece is claimed.
5. w = ctl_.fetch_or(CLOSED, acq_rel) | CLOSED. No worker can enter after this.
6. If active(w) ≠ 0: spin_until(active(ctl_.load(acquire)) == 0), else wait on done_ under m_ with the same predicate.
7. Return true. Every claimed piece was claimed by the caller or by a worker that entered before step 5, and each of those has left (active == 0), so every piece is finished. Workers that never entered never touched next_ or the job fields.
Worker (index, seen):
1. Wait until gen(ctl_) ≠ seen: spin_until, then wake_.wait(m_) with that predicate (as today).
2. enter: w = ctl_.load(acquire); loop: if (w & CLOSED) || index ≥ nw(w): set seen = gen(w) and go to 1. Otherwise compare_exchange_weak(w, w + 1, acquire); on success, seen = gen(w).
3. work(index + 1) reads the job fields. The acquire on the CAS pairs with the caller's release store in step 3, and the fields cannot change while active > 0.
4. left = ctl_.fetch_sub(1, acq_rel) − 1. If (left & CLOSED) && active(left) == 0: lock m_, done_.notify_one(). The release on the decrement publishes the worker's output writes to the caller's acquire in step 6.
5. Go to 1.
A worker that slept through several jobs re-syncs seen to the generation it observed. Entering whatever job is open is correct, because the fields it then reads belong to that job. The 31-bit generation can only alias if one worker stalls across 2^31 jobs between its load and its CAS; even then the CAS only enters the currently open job, which is valid. grow starts new workers at gen(ctl_) (as today), so a thread created for a job enters it when it starts, if the job is still open.
The tid contract is unchanged (caller 0, worker index + 1, index < nw ≤ nt − 1), so per-thread scratch indexing and determinism are unaffected. Fork handling is unchanged: the child builds a new Pool.
What remains: a worker that entered and is then preempted before it leaves still delays the call. That window is a few instructions, against the old whole wake-up path. Finer pieces (C7) shorten the other remaining wait, a preempted worker in the middle of a piece.
Alternatives rejected:
- An atomic-wait or futex wake instead of the condition variable: measured, no gain.
- Per-worker semaphores: same wake latency.
- Longer spinning (OpenMP's 200 ms block time): CPU cost (C9).
- A generation-tagged claim counter, next_ with the generation in its high bits: it lets late workers race on the next job's fields unless the fields are double-buffered. The entry count is simpler and keeps the fields single.
Constraints.
- Determinism: unchanged; pieces are independent and the tid range is unchanged.
- Allocation: none in run.
- ABI: none.
- Spec: no departure. Update the decisions row of 2026-10-15 ("spins 50 µs before sleeping") with: "a call waits only for the workers that entered it".
Tests.
- In tests/c/test_core.cpp, test_concurrent_callers stays. Add test_pool_late_workers: 10 000 calls of a 15-thread parallel_for over 15 one-item pieces, alternating with 300 µs sleeps. Each piece writes tid into a per-call array, and the test asserts that every item was written exactly once and that no tid ≥ nt appears.
- Concurrent callers. 8 user threads each make 20 tv1d prox calls on 512 × 512 at t = 0, each with its own output. Every result must be bit-identical to a serial call.
- Fork. A background thread loops soft prox at t = 8 on 2^20 entries; the main thread forks 20 times; each child makes 5 calls at t = 8 and exits 0 only if its result equals the serial one. Assert 20/20 within 5 s each. Skip the fork part under TSan, which does not support fork with threads.
- Run all of them under the tsan CI job.
As implemented. As designed: ctl_ holds the generation (31 bits), CLOSED, the workers allowed in and the workers inside, and the caller closes the job after work(0) and waits only while workers are inside.
- Tests.
- tests/c/test_parallel.cpp: 10 000 jobs of 15 one-item pieces, every other one after a 300 µs sleep, each item exactly once on a tid below 15; and 8 concurrent callers.
- test_core's concurrent callers: 8 threads at t = 0 on 512 × 512, bit for bit; and 20 forks while another thread keeps the pool busy, each child within 5 s or killed.
- The whole suite passes under ThreadSanitizer, with 0 reports.
- Measured (profile_kernels core, alternating old and new, load average 20):
| Case | Before | After |
|---|---|---|
| Empty 15-thread job, warm | 119–185 µs | 0.6–1 µs |
| Empty 15-thread job, cold median | 91–231 µs | 15–26 µs |
| Empty 15-thread job, cold p99 | 275–891 µs | 43–141 µs |
| Soft, 2^18 entries, t = 15, warm | 116–146 µs | 23–27 µs |
The Python scaling line at 15 threads went from 1.28–1.32 ms to 0.95–1.01 ms. Under that load it says little about efficiency, but nothing regressed.
Acceptance. The prototype passed test_core in Release and under ThreadSanitizer (0 reports), and the concurrency and fork scenarios above.
profile_kernels core cold (C11), median of 200, empty job:
| Threads | Cold: before → after | Warm: before → after |
|---|---|---|
| 15 | 102–107 → 13.7 µs | 1.8 → 2.1 µs |
| 8 | 65 → 5.0 µs | unchanged |
Same case, soft at t = 15, cold:
| Entries | Before → after |
|---|---|
| 2^18 | 97.8 → 56.1 µs |
| 2^20 | 128 → 94.5 µs |
| 2^22 | 304 → 268 µs |
Warm calls stay within ±10%. The scaling line of uv run python bench/bench_python.py ("Parallel scaling, tv1d on 64 rows of 1e4", target ≥ 0.7 × threads) still meets its target.
C7. Grain: pieces scaled to the thread count for prox and jvp¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: S · Depends on: C11 (C6 compounds it)
Problem. Row pieces are row_chunk(n) = kChunk / n rows (driver.hpp:27), 16 384 entries whatever the thread count or the operator's cost. Elementwise pieces are fixed at kChunk = 16 384 entries (driver.hpp:126,277). Parallelism is therefore capped at rows · n / 16384 pieces.
Examples (warm, best of 7): - tv1d on 2048 rows of 16 gets 2 pieces: 66.8 µs at every t ≥ 2, against 145 µs at t = 1. - tv1d on 4096 rows of 16 gets 4 pieces: 82.5 µs at t = 15. - Soft on 2^15 entries gets 2 pieces: 8.0 µs at every t ≥ 2.
Design. In driver.hpp:
constexpr int64_t kPiecesPerThread = 8; // rows
constexpr int64_t kRowPieceMin = 512; // entries per row piece, at least
constexpr int64_t kElemPiecesPerThread = 4;
constexpr int64_t kElemPieceMin = 8192; // entries per elementwise piece, at least
// Rows per piece: about 8 pieces per thread, at least 512 entries and at
// most kChunk entries (the old grain) per piece.
inline int64_t row_grain(int64_t n, int64_t rows, int nt) noexcept {
const int64_t want = ceil_div(rows, kPiecesPerThread * std::max(nt, 1));
const int64_t floor_rows = ceil_div(kRowPieceMin, std::max<int64_t>(n, 1));
return std::max<int64_t>(1, std::min(row_chunk(n), std::max(want, floor_rows)));
}
// Entries per elementwise piece (prox and jvp only): about 4 per thread,
// between 8192 and kChunk, a multiple of 64.
inline int64_t elem_piece(int64_t numel, int nt) noexcept {
const int64_t want = ceil_div(numel, kElemPiecesPerThread * std::max(nt, 1));
return std::min(kChunk, std::max(kElemPieceMin, (want + 63) / 64 * 64));
}
ntis the value after thekParallelMingate (sont = 1gives the old grain).- The row path of
prox_driverandjvp_driverusesparallel_for(nt, p.rows, row_grain(n, p.rows, nt), ...). - The elementwise path of
prox_driverandjvp_driverusesconst int64_t E = elem_piece(p.numel, nt), nch = ceil_div(p.numel, E);withp.for_runs(c0 * E, min(numel, c1 * E), ...). spectral_drivermay userow_grain(k*k, rows, nt); it has no reductions.
Determinism. prox and jvp have no cross-entry floating-point sums: each entry or row is computed by the same code from the same inputs, and the tie count is an integer sum. Their piece size may therefore depend on nt.
The vjp must not change. Its blocks are reduction_blocks(units) with units = ceil(numel / kChunk) (elementwise) or rows, and the rows or chunks of block b are [units·b/nb, units·(b+1)/nb). Its parallel_for runs over blocks with grain 1, as today. Add a comment at vjp_driver saying that its decomposition is part of the results and must not use row_grain or elem_piece.
The constants are tuned. With 2048-entry elementwise pieces and 8 pieces per thread, the shared-counter traffic made soft at 2^16 slower at t = 2 and t = 4 (15.8 → 19.2 µs, 8.3 → 11.7 µs). The values above removed that.
Alternatives rejected:
- A per-operator cost hint, fp_op::row_cost: more API surface for a gain the thread-scaled grain already gives.
- Guided scheduling (decreasing pieces): more counter traffic, and no gain over 8 pieces per thread.
Constraints. - Determinism: as above. - Allocation: none. - ABI: none. - Spec: no departure.
Tests.
- In tests/c/test_core.cpp, test_batching, add tv1d and soft on (2048, 16), (4096, 16) and (1, 2^15): prox and jvp bit-identical across t ∈ {1, 2, 3, 8, 15}.
- vjp gparams bit-identical across the same thread counts.
As implemented. As designed, in prox_driver and jvp_driver, with a comment at vjp_driver that its blocks are part of the result. Contraction off (#105) makes an entry's result independent of how its run is cut, which elementwise pieces that depend on the thread count need.
- Test. test_batching adds soft and tv1d on (2048, 16), (4096, 16) and (1, 2^15), with prox, jvp and vjp gradients bit-identical for t ∈ {1, 2, 3, 8, 15}.
- Measured (warm, best of 7, t = 15, load average 20):
| Case | Before | After |
|---|---|---|
| tv1d, 2^15 entries in rows of 16 | 168–185 µs | 43–56 µs |
| tv1d, 2^16 entries in rows of 16 | 281–287 µs | 77 µs |
| tv1d, 2^18 entries in rows of 16 | 474–532 µs | 286–383 µs |
| soft, 2^15 entries | 13.5–14.6 µs | 7.9–8.3 µs |
The open point, tv1d on 2^16 entries in rows of 16 at t = 2, measured 400 and 462 µs before and 444 and 375 µs after: within the noise of a loaded machine.
Acceptance. profile_kernels core grain (C11), warm, best of 7, t = 15. The prototype numbers are with the constants above.
| Case | Before → after |
|---|---|
| tv1d, 2^15 entries in rows of 16 | 66.8 → 14.9 µs |
| tv1d, 2^16 entries in rows of 16 | 82.5 → 33.1 µs |
| tv1d, 2^17 entries in rows of 16 | 118.9 → 81.4 µs |
| tv1d, 2^18 entries in rows of 16 | 210.9 → 164.1 µs |
| soft, 2^15 entries | 8.0 → 4.4 µs |
| soft, 2^16 entries | 8.3 → 5.8 µs |
| soft, 2^18 entries | 16.6 → 13.8 µs |
| soft, 2^20 and 2^23 entries | unchanged (same pieces) |
One open point: tv1d on 2^16 entries in rows of 16 at t = 2 measured 153 → 173 µs. Recheck it on an idle machine; it must stay within 5% before merging. The scaling line of uv run python bench/bench_python.py does not regress.
C8. Strided operator axis: gather and scatter rows in tiles¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: M · Depends on: C2 (scratch accounting), C7 (grain), C11
Problem. When the operator axis is not the last one (sx ≠ 1), the row drivers copy one row at a time (gather/scatter, driver.hpp:49-55, used at :172, :190, :318, :410, :427). Each element of a row then sits in its own cache line, and for strides of 4096 doubles on its own 16 KiB page.
tv1d (unit weights, auto method) on a (256, 4096) C-order array with axis 0, i.e. 4096 rows of 256 at stride 4096, takes 15.6 ms at t = 1. The same data as 4096 contiguous rows takes 8.6 ms. In isolation, gathering and scattering those 2^20 doubles costs 5.72 ms row by row against 1.43 ms in tiles of 8 rows and 1.22 ms in tiles of 16.
Design. Tiles are used when the RowPlan has nouter ≥ 1 and the last outer dimension has stride 1 in the array being read: x for prox, dx for jvp, g for vjp. That is the layout of x[..., i, j] with the operator on an earlier axis. Rows r and r + 1 then start one element apart. Scatter tiles are used independently, when out (dout, gx) has the same property; otherwise the per-row scatter stays.
- Tile size.
B = clamp(kTileBytes / (n · itemsize), 1, 16)withkTileBytes = 32 KiB; tiles are used only whenB ≥ 4. Examples: n = 256 in f64 gives B = 16, and n = 1024 in f64 gives B = 4. - Scratch.
row_thread_bytesgains2 · round64(kTileBytes): an input tile and an output tile. That is 64 KiB per scratch thread for every row-operator workspace, because the workspace is created fromlikeand may be used later with other strides. With C2, at mostrowsscratch threads. - Tiles do not cross the last outer dimension. With
L = oshape[nouter − 1]andtpl = ceil(L / B), tile t covers linet / tpland rowsj0 = (t % tpl)·Bup tob = min(B, L − j0). Its first row index isr0 = (t / tpl)·L + j0, and its rows arer0 … r0 + b − 1, consecutive in canonical order. The prox and jvpparallel_forruns over tiles, with a grain ofmax(1, row_grain(n, rows, nt) / B)tiles. - Gather. With
off0the offset of row r0 (p.offsets(r0)), rowr0 + kstarts atoff0 + k. The loop isfor i < n: for k < b: tin[k·n + i] = x[off0 + i·sx + k], so each i reads b consecutive elements. In conjugate mode, divide by σ in the same loop. Withkeep, gather directly intows.xs<T>() + r0·n, since the saved rows r0 … r0 + b − 1 are one contiguous b·n block. - Kernel. Rows
k = 0 … b − 1run as today, fromtin + k·n(or the saved rows) intotout + k·n(orws.zs + (r0 + k)·n), with codes atws.code + (r0 + k)·n. - Scatter. If out's last outer stride is 1:
for i < n: for k < b: od[offo0 + i·so + k] = tout[k·n + i], with the conjugate formx − σ·resapplied in this loop. Otherwise scatter per row. - vjp. Iterate the rows of each fixed block (
[rows·blk/nb, rows·(blk+1)/nb)) in ascending order, cutting tiles at block boundaries as well as at line boundaries. The accumulation order intoacc + blk·Wis then the same ascending row order as today, so gradients are bit-identical.
Alternative rejected: transposing the whole array into scratch. That is O(numel) workspace, which breaks the "4 rows per thread" scratch model.
Constraints.
- Determinism: each row is computed by the same kernel from the same contiguous values, and vjp blocks and order are unchanged, so results are bit-identical to today.
- Allocation: +64 KiB per scratch thread, sized in build_workspace.
- ABI: none.
- Spec: no departure.
Tests. In tests/c/test_core.cpp, for every row operator in derivative_cases(n) with n ∈ {12, 257}, on shape (n, 37) with axis 0, and shape (n, 3, 1003) with axis 0 (lines of 1003 rows, not a multiple of B): prox, prox_conj, jvp and vjp (including gparams and ggamma) bit-identical to the same data transposed to contiguous rows. Test at t ∈ {1, 8}, and with keep and without. Also check an out with a different layout (C order while x is strided) to exercise gather tiles with per-row scatter.
As implemented. As designed, in the prox, jvp and vjp row paths of driver.hpp.
- Plan. make_tiling (B = min(16, 32 KiB / (n · itemsize)), tiles only for B ≥ 4) and tile_layout decide, from the RowPlan, whether the read and the written array qualify.
- Paths. Each driver gets a tiled branch in front of its per-row loop, which stays as it was for every other layout:
- prox: gathers into the tile or the saved rows, then runs the kernel row by row, then scatters by tile or by row; the conjugate's x − σ·res is formed in the scatter;
- jvp: tiles dx;
- vjp: tiles g within each fixed block, cut at block and line boundaries, so each block sums its rows in ascending order as before.
- In place is safe: a whole tile is read before any of it is written.
- Scratch. RowBuffers gained the two tiles after the kernel scratch, and its constructor takes the operator to place them.
- Measured (profile_kernels core strided, t = 1, alternating master and this branch):
| Case | Before | After |
|---|---|---|
| (256, 4096) along axis 0 | 13.8 ms | 8.9 ms (target ≤ 11) |
| (4096, 256) along axis −1 | 8.0 ms | 8.0 ms |
wsbytes shows the 65 536 bytes added per scratch thread.
- Test. test_strided_tiles covers every row operator of derivative_cases(n) for n ∈ {12, 257}, on (n, 37) and (n, 3, 1003) along axis 0; (n, 3, 103) for n = 257, to keep the test short. It checks prox, the conjugate, jvp and vjp (gamma and parameter gradients) bit for bit against the same rows stored contiguously, at t ∈ {1, 8}, with the output in the strided layout and in C order, and in place without keep.
Acceptance. profile_kernels core strided (C11), t = 1:
- the axis-0 case drops from 15.6 ms to ≤ 11 ms (target; the contiguous case is 8.6 ms);
- the contiguous case is unchanged (±2%).
The tv1d line of uv run python bench/bench_python.py does not regress.
C9. Spin time as an environment knob¶
- Kind: speed · Severity: low · Target: 1.0.x · Effort: S · Depends on: C6 (latency after sleep is then bounded), C11
Problem. Every worker spins for kSpin = 50 µs after each job (parallel.cpp:84), and so does the caller while it waits. One thousand empty 15-thread jobs, one every 1 ms, burned 1.07 s of process CPU in 1.35 s of wall time, about 0.8 cores for no work. Callers that interleave short parallel calls with other work (a Python solver loop with NumPy in between) pay this on every call. There is no way to turn it off or lengthen it.
Design.
- kSpin becomes spin_time(), read once from the environment variable FASTPROX_SPIN_US: a decimal integer of microseconds, clamped to [0, 100 000], default 50, with invalid values taken as the default. It is a static local, read like default_threads().
- spin_until takes the duration. With 0 it checks the predicate once and returns, so workers sleep at once and the caller blocks on done_.
- No C API function: an environment knob needs no ABI change, and the value must be fixed before the pool starts.
Contract text, docs/api.md "Threads and determinism", replace "The pool's threads spin for about 50 µs after a call before sleeping." with:
The pool's threads spin for 50 µs after a call before sleeping, so that back-to-back calls find them awake. The environment variable
FASTPROX_SPIN_US, read once, sets this time in microseconds, from 0 (sleep at once: the least CPU, but about 10–100 µs more latency on the next call) to 100 000.
Alternative rejected: adapting the spin to the observed gap between calls. Heuristic and hard to test; the knob is enough.
Constraints.
- Determinism: unaffected.
- Allocation: none.
- ABI: none.
- Spec: no departure. Extend the 2026-10-15 decisions row: "the spin time is FASTPROX_SPIN_US (default 50)".
Tests. In tests/c/test_core.cpp, run the concurrency tests of C6 a second time in a child process started with FASTPROX_SPIN_US=0, asserting the same results as the default run.
As implemented. As designed, with spin_time() a static local.
- Test. CTest runs test_parallel a second time with FASTPROX_SPIN_US=0 (parallel_nospin).
- Measured. With 0, process CPU fell from 0.82–0.92 s to 0.15–0.19 s. That misses the ≤ 0.1 s target: what remains is the 15 workers woken by each job, who find it already closed, at about 10 µs a wake-up on a machine under load average 20. With 0 the cold empty job stays at 4–21 µs.
Acceptance. profile_kernels core spin (C11):
- with FASTPROX_SPIN_US=0, the process CPU it prints falls from 1.07 s to ≤ 0.1 s;
- with the default setting, the cold and grain cases are unchanged.
C10. Guard against a nested parallel_run from inside a job¶
- Kind: safety · Severity: low · Target: 1.0.0 · Effort: S · Depends on: C6 (same file)
Problem. parallel.hpp:31 says fn "must not call parallel_run", but nothing enforces it.
- From a worker thread, a nested call's
try_lock(run_m_)fails and the call runs serially, which is fine. - From the caller thread (tid 0),
try_lockis called on astd::mutexthat this thread already owns. That is undefined behaviour (glibc and macOS return EBUSY, a debug or checked runtime may abort).
Today nothing reaches it: the outer image path passes inner = 1 (driver.hpp:626), and parallel_run returns before the pool when nt ≤ 1. But any future kernel that calls parallel_for(c.nt, ...) with c.nt > 1 inside a row or outer image task would hit it.
Design.
- In parallel.cpp, add thread_local bool t_in_job = false;.
- Set it in Pool::work around the piece loop (for the caller's work(0) and each worker's work(index + 1)), with a small RAII guard that restores the previous value.
- At the top of parallel_run, after the count ≤ 0 check: if (t_in_job) { fn(ctx, 0, 0, count); return; }.
- The nested call then runs serially as tid 0 of its own scratch, which is what a failed try_lock does today.
- Add to parallel.hpp: "A call from inside fn runs serially on the calling thread, with tid 0; fn's tid indexes only scratch that belongs to its own call."
- In debug builds, assert(!t_in_job || nt <= 1) documents that nested calls are not expected to ask for threads.
Alternative rejected: a recursive mutex. It would let the nested call post a second job into a pool whose workers are busy, and deadlock.
Constraints.
- Determinism: unchanged (a serial call gives the same result).
- Allocation: none (a thread_local bool, constant-initialised).
- ABI: none.
- Spec: no departure.
Tests. In tests/c/test_core.cpp, link a test-only translation unit against src/core/parallel.cpp. An outer parallel_for(8, 64, 1, ...) whose body calls parallel_for(8, 16, 1, ...) must complete with every inner item visited once and every inner tid 0. Run it in Release, in Debug with -D_GLIBCXX_ASSERTIONS (Linux) and in the tsan job.
As implemented.
- The thread-local flag is set for the duration of Pool::work, for the caller and for each worker.
- The debug assert was left out: the test this item specifies, a nested parallel_for(8, …), would trip it.
- The test is tests/c/test_parallel.cpp, which compiles parallel.cpp in, because the shared library hides its internal symbols. It checks that every inner item is visited once, with tid 0, and that the pool works afterwards.
- On macOS the old code also passes. There, try_lock on a mutex the thread already holds returns busy, and ThreadSanitizer does not report it. The undefined behaviour is real but does not fail on this platform.
Acceptance. The test passes in all three builds. The cold case of profile_kernels core (C11) is unchanged (one thread-local load per call).
C11. A core section in bench/profile_kernels.cpp¶
- Kind: speed (tooling) · Severity: low · Target: 1.0.x · Effort: S · Depends on: none
Problem. C2 and C4–C9 need repeatable measurements of the core: workspace bytes, elementwise throughput, derivative cost, pool latency, grain and strided rows. bench/bench_python.py adds Python overhead of several µs per call, and bench/profile_kernels.cpp profiles only 1D TV.
Design.
- Add a mode core to bench/profile_kernels.cpp: ./profile_kernels core [case], with case one of wsbytes, elem, deriv, cold, grain, strided, spin, or all of them when omitted. The existing positional arguments keep their meaning when the first one is not core.
- Build as the file's header says. Cases that need internal functions (cold's empty job, spin) compile src/core/parallel.cpp into the binary with -Isrc, so the header line becomes c++ -O2 -std=c++17 bench/profile_kernels.cpp src/core/parallel.cpp -Iinclude -Isrc -Lbuild/c -lfastprox -ldl ....
- Inputs are normal random from std::mt19937_64 seed 1, f64. Every case prints one line per configuration.
The cases:
- wsbytes. Bytes allocated by fp_workspace_create (the malloc statistics around the call: malloc_zone_statistics on macOS, mallinfo2 on glibc) for tv1d, one row of n ∈ {2^16, 2^20, 2^23}, t ∈ {1, 0}. Also the best-of-5 fp_prox time with that workspace.
- elem. ns per entry, t = 1, 2^16 entries, best of 3000 calls. Covers soft, weighted soft (weights 0.7, length n), elastic net, huber, box [−0.5, 0.5], nonneg, mcp and l0, each out of place and in place. Also a hand-written soft loop as the floor.
- deriv. 256 × 1000, kept structure. For soft, nonneg and huber at t ∈ {1, 15}: prox, jvp (dx = g, dgamma = 0.3, dparams NULL), vjp with ggamma and gparams, and vjp with both NULL. µs, best of 30.
- cold. Two parts:
- empty parallel_for(nt, nt, 1, f), nt ∈ {4, 8, 15}: warm median of 20 000, and cold median and p99 of 300 (300 µs sleep before each);
- soft prox on 2^18, 2^20 and 2^22 entries at t ∈ {1, 4, 8, 15}: warm and cold medians of 200.
- grain. Best of 7, warm, at t ∈ {1, 2, 4, 5, 8, 15}:
- soft on 2^15, 2^16, 2^18, 2^20 and 2^23 entries;
- tv1d (unit weights, auto method) on N/16 rows of 16 for N ∈ {2^15, 2^16, 2^17, 2^18}.
- strided. tv1d, t = 1, best of 30: shape (4096, 256) along axis −1, and shape (256, 4096) along axis 0.
- spin. 1000 empty 15-thread parallel_for jobs with a 1 ms sleep between them: process CPU time (std::clock) and wall time. Run it once as is and once with FASTPROX_SPIN_US=0.
Constraints. - Bench code only. - Clean room: the existing proxTV rule of the file is unchanged.
Tests. None; CI only builds it, by adding the build line to the c job.
As implemented. As designed. Every case of ./profile_kernels core runs in about 12 s on the M5 Pro. The wsbytes case reads malloc_zone_statistics on macOS and mallinfo2 on glibc. CI builds the profiler in the c job (not on Windows: the file uses dlopen) and runs its elem case. The before and after numbers of C6, C7 and C9 below come from it, with master's library and pool built in a separate worktree. The runs alternated old and new on a machine under heavy background load (load average 20–28), which is what made the C6 gain visible.
Acceptance. ./profile_kernels core runs every case in under 60 s on the M5 Pro. Its pre-change output reproduces the "before" numbers quoted in C2 and C4–C9 within ±15%, given the load caveat in the notation.
K. Elementwise and row kernels¶
Scope: src/ops/pointwise.cpp, projections.cpp, group.cpp, radial.cpp,
nonconvex.cpp, isotonic.cpp, hyperplane.cpp, tv1d.cpp, tv1d_l2.cpp,
with src/core/op.hpp, as of master d67de78. Each reproducer states its
operator, input and γ inline, so it can be rerun from Python.
Notation: ε = 2⁻⁵² (double), ldexp/frexp from <cmath>. "Power-of-two
scaling" means multiplying by t = 2^k, which is exact unless a result is
subnormal, so it changes no bit of a result that does not over- or underflow.
New benchmark section: kernel_throughput() in bench/bench_python.py¶
The speed criteria below (K6, K12) and the no-regression checks of the
correctness items (K1, K2, K7, K8, K11) read from a new section. It is added
to bench/bench_python.py and called from __main__ after
simplex_throughput(). It prints "Row and elementwise kernels, one thread,
ns per entry" and then one line per setup, all with threads=1, timed with
the file's best(..., repeat=3) and divided by the number of entries:
| Label | Setup (rng = np.random.default_rng(4), n = 10⁶) |
|---|---|
soft weighted |
fp.Soft(0.3, w).prox(x), x = rng.normal(size=n), w = rng.uniform(0, 2, n) with every 10th weight 0 |
lq q=0.9 far / lq q=0.9 near |
fp.Lq(1.0, 0.9).prox(x), x = τ·rng.uniform(3, 10, n) and x = τ·(1 + rng.uniform(1e-9, 1e-3, n)), with τ = β + q β^(q−1), β = (2(1−q))^(1/(2−q)) |
lq q=0.999 far / lq q=0.999 near |
the same with q = 0.999 |
l2_ball prox / jvp / vjp |
fp.L2Ball(1.0) on one row x = rng.normal(size=n) (outside), prox(x, keep=True) once outside the timer, then prox(x), jvp(ctx, dx) and vjp(ctx, g) with dx and g normal |
prox_l2 groups of 8 |
fp.group_soft_threshold(x, 0.5, groups=np.arange(n) // 8) |
soc |
fp.SOC().prox(x) on one row of n |
hyperplane box |
fp.project_hyperplane(x, a, 0.3*n, 0.0, 1.0), a = rng.uniform(0.5, 1.5, n), x = rng.normal(size=n) |
sum_largest k=10 |
fp.prox_sum_largest(x, 1.0, 10) |
isotonic random walk |
fp.isotonic(np.cumsum(rng.normal(size=n))) |
Report the numbers before and after each change in its pull request. "No regression" means within 2% of the value before, best of three runs of the script on Apple M5.
Shared design for K2, K7, K8, K9: scale only where the plain sum fails¶
Four kernels form sums of squares or of entries that overflow (or, for squares, underflow) long before the quantity they compute does. They share one rule, so that the fast path is unchanged bit for bit:
- Compute the plain sum as today, in the same order.
- Accept it when it lies in a safe range, which guarantees that no
partial sum overflowed and that terms lost to underflow are below
rounding. For a sum of squares S of n terms the range is
0x1p-968 <= S && S <= DBL_MAX. The partial sums of nonnegative terms are nondecreasing, so a finite S means none overflowed. Each square that rounds into the subnormal range is off by at most 2⁻¹⁰⁷⁵, so with S ≥ 2⁻⁹⁶⁸ the total loss is at most n·2⁻¹⁰⁷ relative, below rounding for n < 2⁵⁰. A NaN fails both tests and goes to the slow path, which keeps it. - Otherwise (rare), take M = max |xᵢ| in a second pass. Then choose
t = 2^(−k) with
frexp(M, &k), so that M·t ∈ [0.5, 1), and redo the sum on t·xᵢ. The result is reported in scaled form.
This is the idea behind the reference BLAS dnrm2: J. L. Blue, "A portable
Fortran program to find the Euclidean norm of a vector", ACM TOMS 4(1), 1978,
15–23, and E. Anderson, "Algorithm 978: Safe scaling in the Level 1 BLAS",
ACM TOMS 44(1), 2017. It is cited in prose only. The difference here is that
the scale is a power of two chosen after the fact, not accumulated during
the pass, so the common path costs nothing extra. tv1d_l2.cpp already uses
the same power-of-two normalisation for its prox (lines 242–253).
K1. Jacobian 0 instead of 1 where the threshold is 0 and x = 0¶
- Kind: correctness · Severity: high · Target: 1.0.0 · Effort: S · Depends on: none
Problem. The tie-breaking rules at kinks treat |x| = τ as thresholded, and
they are also applied when τ = 0. At τ = 0 the map is the identity, which
docs/api.md states twice: "a zero weight leaves the entry unchanged" and "a
map that is the identity (γwᵢ = 0 or λ = 0) has none [kinks]". The entry
x = 0 still gets the Jacobian 0. Soft(1, [0, 1]) at x = [0, 2] gives
jvp(ones) = [0, 1], where [1, 1] is expected. Soft(0), SCAD(0, 3.7),
MCP(0, 3), L0(0), L1Box(0, -1, 1) and LogPenalty(0, 1) at x = [0, 0.5]
all give [0, 1]. GroupL2(1, [0,0,1,1], [0, 1]) at [0, 0, 3, 4] gives 0 on
the unpenalised group. Elastic net with l1 = 0 has the same fault: it returns
0 where 1/(1+γ l2) is expected. Lq is already right (nonconvex.cpp:262).
In practice this blocks the gradient of an unpenalised coefficient (an
intercept, or a zero weight) that starts at 0, which is the usual start of
unrolled ISTA.
Design. There is one rule for every map: a threshold τ with !(τ > 0)
selects the identity piece, or the identity's analogue for the operator.
deriv1 must keep equal to deriv_piece(x, piece_of(x)) (the ElemOp
contract in op.hpp:308). The sign s = sgn(x) is 0 at x = 0, so every
parameter term of the identity-adjacent piece vanishes there by itself.
That is the correct one-sided limit: z(x = 0) = 0 for every τ ≥ 0, so
dz/dw = dz/dγ = 0. Per operator:
- Soft (
pointwise.cpp:61-63).piece_ofreturns id 1 when!(tau > 0.0) || fabs(x) > tau. Piece 1 with s = 0 gives j = 1, dg = 0 and pv = 0. - ElasticNet (
:106). The condition becomes!(gamma*l1 > 0.0) || fabs(x) > gamma*l1. With s = 0 the branch gives j = 1/d, dg = −(l2·z)/d = 0 and pv = (0, 0), which is correct at x = 0. - L1Box (
:363-367). The last line returns{(!(tau > 0.0) || fabs(x) > tau) ? 1 : 0, sgn(x)}. The bound pieces 2 and 3 do not change. - L0 (
nonconvex.cpp:113-127).deriv1usese.j = (!(t > 0.0) || a > t || (a == t && tie == FP_TIE_NONZERO)) ? 1 : 0, andpiece_ofreturns id 1 when!(thr > 0.0) || fabs(x) > thr. - SCAD and MCP (
:411,:494). The zero piece needsc * lambda > 0.0, notc > 0.0:if (ax <= c * lambda && c * lambda > 0.0) id = 0;. With λ = 0, x = 0 then falls to id 1 for SCAD (j = 1, dzdc = 0, pv = 0) and to id 1 for MCP (j = b/(b−c), which is wrong). MCP must also take the identity piece when!(lambda > 0.0): set id = 2 directly. (As implemented: not when c = 0 with λ > 0, where the linear piece gives j = b/b = 1 and the right one-sided dz/dc, which id 2 would zero.) - LogPenalty (
:576-586). Addif (lam == 0.0) return branch_deriv(fabs(x), sgn(x), gamma);before the!ontest. At λ = 0 this is j = 1, dz/dλ = −γs/(ε+|x|) and dz/dε = 0, the derivative of the nonzero branch at its start.piece_ofalready returns 1. - GroupL2 (
group.cpp:83-92). A group with τ_g = 0 is live. The prox usesscale[g] = !(tau > 0.0) ? 1.0 : (s > tau ? 1.0 - tau / s : (s == s ? 0.0 : s)), which stays exact and keeps NaN groups NaN because x·1 = NaN, andcode[i] = scale[group(i)] > 0.0then marks the group live. Injvpandvjp, a live group with s = 0 (only possible at τ = 0) takes a = 1, b = 0 and adds nothing to acc. Without that guard, cc = 0/0. - TreeL2 (
group.cpp:241-372). The ℓ1 stage tests!(tau1 > 0.0) || fabs(v) > tau1injvp(:294) andvjp(:363), withsg = sgn(v), which is 0 at v = 0, instead ofv > 0 ? 1 : -1. Each shrink treats!(tau > 0.0)as live with a = 1. When s = 0 the update coefficient is 0; this is safe with the K5 form. - Group Huber, SCAD and MCP inherit the fix: at x_g = 0 they use the
scalar map's j(0) (
radial.cpp:150-152).
What kinks and jacobian_diag should report: no change to kink1 or
group_kinks. Every one of them already returns 0 when τ ≤ 0 (tau > 0,
t > 0, lam > 0, c > 0 && lambda > 0). So ctx.kinks() is 0 there, a
branch override at such an entry is ignored, and ctx.jacobian_diag() reads
1. Before this fix it read 0, which contradicted the empty kink mask.
Contract text. In the docs/api.md kink table, the soft/elastic-net row
becomes "|xᵢ| equal to a positive threshold | thresholded: 0". Add a sentence
under the table: "A zero threshold (λ = 0, wᵢ = 0, c_g = 0 or γ = 0) is not a
kink: the map is the identity there and its Jacobian is 1 (1/(1+γ l2) for the
elastic net), also at xᵢ = 0." Update the header comment of
pointwise.cpp:16-19 in the same way.
Alternative rejected: treating τ = 0, x = 0 as a kink with both sides available. The map is linear there, so no second piece exists.
Constraints.
- Determinism: unaffected, since the change is per entry or per group.
- Allocation: none.
- ABI: unchanged.
- Clean room: nothing new.
- Spec: no departure. docs/spec.md Section 7 defines the Jacobian "where the
prox is differentiable", and it is differentiable here. This is a
documentation clarification plus a CHANGELOG Fixed entry.
Tests.
- tests/python/test_kinks.py, new test test_zero_threshold_is_identity,
parametrised over Soft(1, [0, 1]), Soft(0), ElasticNet(0, 0.5),
L1Box(0, -1, 1), L0(0), Lq(0, 0.5), SCAD(0, 3.7), MCP(0, 3),
SCAD(0.5, 3.7, [0, 1]), MCP(0.5, 3, [0, 1]) and LogPenalty(0, 1), at
x = [0, 0.5]. Assert:
- jacobian_diag() equals the identity's (1, or 1/(1+γ l2) for the
elastic net) on every zero-threshold entry, exactly;
- kinks(1e-12) is 0 there;
- the vjp equals the jvp transpose (adjointness, rtol 1e-14);
- the parameter jvp at x = 0 is 0, and agrees with one-sided finite
differences in θ (h = 1e-7, atol 1e-6).
- The same test also covers GroupL2(1, [0,0,1,1], [0, 1]) at [0, 0, 3, 4],
where jvp(ones) must be [1, 1, 0.968, 1.024], and
TreeL2([[0, 1]], [0.0], 0.0) at [0, 0], where jvp(ones) must be
[1, 1].
- tests/c/test_core.cpp, derivative_cases: add a soft case with a zero
weight entry and an x containing an exact 0, so that the C-level
finite-difference and adjoint checks run at τ = 0.
Acceptance. uv run pytest -q tests/python/test_kinks.py tests/python/test_operators.py
and ./build/c/tests/c/test_core pass. Speed: in
uv run python bench/bench_python.py, the soft weighted line of
kernel_throughput() and the fp.soft_threshold(x) line of overhead()
show no regression. The piece_of change adds one compare inside an
existing branch.
K2. Squared norms overflow or underflow (L2Ball, GroupL2, TreeL2, GroupRadial, SOC)¶
- Kind: correctness · Severity: high · Target: 1.0.0 · Effort: M · Depends on: none (K5 and K12 build on it)
Problem. Five kernels take √(Σxᵢ²) without scaling:
projections.cpp:234-237 (L2Ball prox, jvp and vjp), group.cpp:78-87 and
:252 (GroupL2, TreeL2), radial.cpp:104-112 (GroupRadial) and
radial.cpp:206-213 (SOC).
| Call | Output | Expected |
|---|---|---|
project_l2_ball([1e200, 1e200], 1) |
[0, 0] (jvp 0) | [0.70710678, 0.70710678] |
project_l2_ball([1e-200]*2, 1e-201) |
x, outside the ball | 0.0707·x |
prox_l2([1e-200]*2, 1e-201) |
[0, 0] | 0.92929·x |
GroupL2(1, None, [0])([1e-170]*2) (identity) |
[0, 0] | x |
SOC()([0, 1e200, 1e200]) |
[inf, nan, nan] | [7.0711e199, 5e199, 5e199] |
SOC()([1e300, 1e200, 1e200]) |
[inf, nan, nan] | x (inside) |
SOC()([-1e-201, 1e-200, 1e-200]) |
[0, 0, 0] | [6.571e-201, 4.646e-201, 4.646e-201] |
GroupHuber(1, 1)([1e200]*2), GroupSCAD(1, 3.7) |
[nan, nan] | ≈ x |
GroupHuber(1, 1)([1e-170]*2) |
[0, 0] | [5e-171, 5e-171] |
Design. Add one helper to src/core/op.hpp:
// ||v|| over the entries the accessor f(k), k < len, visits, as m / t with t
// a power of two (t = 1 on the fast path) and m = ||t v|| in [0.5, sqrt(len)]
// on the slow path. NaN in v gives m = NaN.
struct ScaledNorm { double m, t; double value() const noexcept { return m / t; } };
template <class F>
inline ScaledNorm scaled_norm(int64_t len, F f, double plain_sumsq) noexcept;
plain_sumsq is the sum that the caller has already accumulated in its
existing pass, so the fast path costs one range test. The fast path is
0x1p-968 <= S && S <= DBL_MAX, which returns {sqrt(S), 1.0}. Otherwise
the helper runs a pass for M = max |f(k)|. If M is 0 it returns {0, 1}. If
M is NaN it returns {NaN, 1}. Otherwise it sets frexp(M, &k),
t = ldexp(1.0, -k) and runs a pass for Ŝ = Σ (t·f(k))², which lies in
[0.25, len], and returns {sqrt(Ŝ), t}. All passes run in index order,
which keeps results deterministic.
Each consumer then uses the scale-free quantities û = t·x/m (the unit vector), τ·t/m (the ratio τ/s) and r·t (a radius in scaled units). These never overflow where they are used:
- L2Ball prox. Inside when
m <= r * t(r·t = inf means inside; r·t = 0 with m > 0 means outside, both correct). Otherwise c = (r·t)/m and zᵢ = xᵢ·c. Outside means r·t < m ≤ √n, so c does not overflow. - L2Ball jvp and vjp. The same norm. Then ud = Σ (t xᵢ) dxᵢ / m and uᵢ = (t xᵢ)/m: dzᵢ = c (dxᵢ − uᵢ ud) + dr uᵢ, gxᵢ = c (gᵢ − uᵢ ug) and acc[0] += ug. The dot product of x̂ = t x with dx cannot overflow through x, since |x̂ᵢ| < 1.
- GroupL2. The prox pass already accumulates
scale[g](Σ over each group). After it, run one loop over the groups. If any group fails the safe range, call a per-group slow path: pass 2 writes M_g into the second G-double array of the scratch, whichscratch_bytesalready sizes at 2G doubles plus G bytes; it turns M_g into t_g; pass 3 re-sums (t_g xᵢ)² for the flagged groups only. Then live meansm_g > tau * t_g(or τ = 0, as in K1), andscale_g = 1 - (tau * t_g) / m_g.norms_dotsin jvp and vjp does the same, using b_g =/m_g = , cc = (τ t_g)/m_g, and dz_i = a_g dx_i + coef_g * (t_g x_i) / m_g. This needs t_g per group in jvp and vjp as well:scratch_bytesgrows from 2G·8 + G to 3G·8 + G bytes.value_rowneeds no change (it is not in scope here; it inherits the helper if convenient). - GroupRadial (
radial.cpp). The same per-group norm, andscratch_bytesbecomes 3G doubles (nrm, dot, t). The prox computes s = m/t. When s is finite, scale = φ(s)/s as now. When s overflows (the true group norm is above DBL_MAX, so entries are near 1e308), it uses scale = φ(DBL_MAX)/DBL_MAX: φ(s) = s − κ beyond the last breakpoint, so the error is κ/DBL_MAX, below rounding unless the parameters themselves are near DBL_MAX. The derivatives take s = m/t only for the scalar map'sderiv(j, φ_θ and φ_step are evaluated at s), and use the scale-free form for the rest: dz = (φ/s) dx + [(j − φ/s)· + dφ]·u, with u = t x/m. So b_g multiplies (t_g xᵢ)/m_g, not xᵢ/s. - SOC.
regioncomputes the scaled norm of y = x[1:] (fast path from its existing q). The keep test ism <= x0 * t. The zero test ism <= -x0 * t. On the mid branch, ρ = 0.5·(m + x0 t)/m, the scale of y, is scale-free; z₀ = 0.5·(m + x0 t)/t; zᵢ = xᵢ·ρ. Since |x0 t| < m ≤ √n there, nothing overflows.apply(the jvp and vjp) uses uᵢ = (t xᵢ)/m, uv = Σ (t xᵢ) vᵢ / m and r = ρ. - TreeL2.
norm(w, g)becomesscaled_norm(len, [&](k){ return w[idx[ptr[g]+k]]; }, q). The shrink factor is a = 1 − (τ t)/m, live when m > τ t. jvp and vjp use the K5 form below, in which every quantity is scale-free.
Constraints.
- Determinism: the passes run in a fixed order, and the slow-path decision
depends only on the row.
- Allocation: none in calls. build_workspace sizes GroupL2 at
3G·8 + G bytes per thread (was 2G·8 + G) and GroupRadial at 3G·8 (was 2G·8);
the others are unchanged.
- ABI: unchanged.
- Clean room: Blue 1978 and Anderson 2017 are cited in prose in the
op.hpp helper comment and in the headers of the five kernels.
- Spec: no departure; spec Section 10 already asks for accuracy over the
double range. The fast path is bit-identical to today.
Tests.
- tests/python/test_operators.py, new test_norm_scaling. For L2Ball(0.7),
GroupL2 (GROUPS), SOC, GroupHuber, GroupSCAD and TreeL2, and
t ∈ {2⁻¹⁰⁶⁰, 2⁻⁶⁰⁰, 2⁻⁴⁰⁰, 2⁴⁰⁰, 2⁶⁰⁰, 2¹⁰⁰⁰}, check positive
homogeneity on random x: prox(t x; t r or t λ) == t prox(x; r or λ) with
rtol 8ε (radii and thresholds scaled as well). For the jvp in x, check
jvp(t x; t θ)(dx) == jvp(x; θ)(dx), rtol 1e-14, atol 0.
- Add the table above as exact-value regressions (rtol 1e-15).
- fuzz/fuzz_ops.cpp: a new magnitude mode multiplies the row by 2^k,
k ∈ [−1070, 1020], for cases 3, 6, 25, 26–28 (group radial) and the tree
case. The invariant: output finite whenever the expected output is
finite; the existing nonexpansiveness check still holds.
As implemented.
- The helper. One helper, safe_group_norms, with safe_norm and with_norm built on it in core/op.hpp, returns the norm as m / t.
- t = 1 except where the norm itself exceeds the largest double or is subnormal (there 1/‖x‖ overflows). Every other case only needs an accurate s, since x/s, r/s and ⟨x, v⟩/s involve no squares.
- The consumers read the norm through (t·xᵢ)/m, (τ·t)/m and ⟨t·x, v⟩/m, which reduce to the plain formulas, bit for bit, when t = 1. The group derivatives form uᵢ = (t·xᵢ)/m entry by entry, because a per-group coefficient 1/s overflows for subnormal groups.
- Speed. with_norm calls the consumer's body with the constant t = 1 on the fast path, so the products by t fold away, and runs the rest out of line. In C, on rows of 8, the cost is:
- +5% for the ℓ2 ball, from the one range test per row;
- about 0% for the cone and the ℓ2 prox.
A first version with a runtime t cost 5–12%. Long rows are unchanged.
- Subnormal largest entry. The scale 2^-e would overflow there, so it is capped at 2^1022. The fuzz harness found this: group Huber of 4.3e-316 gave NaN.
- Bundled items. K5 (TreeL2's τ/s³) and K10 (Huber's c²) are included, since the homogeneity tests need them.
- Where the tests live. They are in tests/python/test_norm_scaling.py, not test_operators.py, and include subnormal inputs and norms above the largest double. The fuzz harness scales these operators' inputs by 2^±(600–1000), in the mode T5 added.
Acceptance. uv run pytest -q tests/python/test_operators.py tests/python/test_radial.py
pass, plus 10⁷ fuzz executions without a report. Speed: in
uv run python bench/bench_python.py, the l2_ball prox,
prox_l2 groups of 8 and soc lines of kernel_throughput() show no
regression, since the fast path adds one compare per row or group.
K3. Log penalty: overflow of (|x|+ε)² and cancellation in the root¶
- Kind: correctness · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: none
Problem. nonconvex.cpp:547-549 (mag) and :603-606 (branch) form
disc = (a+ε)² − 4λ̃ and r = ½((a−ε) + √disc), with λ̃ = γλ.
The square overflows for a > 1.34e154, which gives r = inf and df = inf, so
the entry maps to 0. LogPenalty(1, 1) at [1e200, 1e154, 1e155] returns
[0, 1e154, 0]; expected ≈ [1e200, 1e154, 1e155].
For a < ε the sum cancels:
| λ̃ | ε | x | Output | Exact (80-digit Decimal) | Error |
|---|---|---|---|---|---|
| 1e-20 | 1 | 3e-20 | 0 | 2.0000000000000005e-20 | 100% |
| 1e-14 | 1 | 1.1e-14 | — | — | 5.5% |
| 1e-10 | 1 | 1e-9 | — | — | 2.1e-8 |
| 1e-6 | 1e3 | 2e-9 | — | — | 1.1e-5 |
Design. Add one helper used by mag, branch and therefore
breakpoint:
// The positive root r of z^2 + (eps - a) z + (lam - a eps) = 0 (stationarity
// z - a + lam / (eps + z) = 0), or 0 when there is none (disc < 0) or it is not
// positive. a >= 0, eps > 0, lam >= 0.
double log_root(double a, double lam) const noexcept;
- Normalise by a power of two. The problem is homogeneous: replacing
(a, ε, r) by (t a, t ε, t r) and λ̃ by t² λ̃ preserves it. The fast path is
S = a + ε ∈ [2⁻⁵⁰⁰, 2⁵⁰⁰] and λ̃ ≤ 2¹⁰⁰⁰, with t = 1. Otherwise
frexp(S, &k), t = 2^(−k), â = t a, ε̂ = t ε, λ̂ = t²·λ̃ computed asldexp(lam, -2k). If λ̂ overflows it is +inf, so disc < 0 and the result is 0, which is correct because then 4λ̃ > S². If it underflows it is negligible against Ŝ² ≥ ¼, also correct. Return r̂/t. - Discriminant. D = Ŝ² − 4λ̂, with Ŝ ∈ [0.5, 1) on the slow path. If !(D ≥ 0), return 0. Let q = √D.
- Root in two regimes, with no subtraction of nearly equal terms:
- â ≥ ε̂: r̂ = ½((â − ε̂) + q). Both terms are ≥ 0.
- â < ε̂: r̂ = 2·fma(â, ε̂, −λ̂) / ((ε̂ − â) + q). The denominator is a sum
of positive terms, and the numerator is the exactly rounded â ε̂ − λ̂.
This is the same root, multiplied by the conjugate:
((a+ε)² − 4λ − (a−ε)²) / (2((ε−a) + q)) = 2(aε − λ)/((ε−a) + q).
The numerator's sign carries the "no positive root" case (r̂ ≤ 0 → 0).
std::fmais correctly rounded on every IEEE platform (in software where there is no instruction), so this stays bit-identical across machines. It is an explicit call, not compiler contraction, so the no-fast-math rule holds. magkeeps its decision on df = r (r/2 − a) + λ̃ log1p(r/ε). For huge inputs, compute it in the normalised units (all three terms scale by t², so the sign and the df == 0 tie test are unchanged), to avoid r·(r/2 − a) overflowing for a > 1e154.- Derivatives (
branch_deriv,:588-600). With u = ε + r, write κ = (λ̃/u)/u instead of λ̃/(u·u): u·u under- or overflows for ε < 1e-154 or r > 1e154, while on the selected branch κ ≤ 1 (second-order condition), so λ̃/u ≤ u never overflows. Then den = 1 − κ, j = 1/den, dg = −λ s/u/den, pv[0] = −γ s/u/den and pv[1] = κ s/den.
Alternative rejected: the q-form √D = S·√((1 − ρ)(1 + ρ)), ρ = 2√λ̃/S, used everywhere. It costs a sqrt more per entry and is no more accurate than normalisation.
Constraints. - Determinism: yes; fma is correctly rounded. - Allocation: none. ABI: unchanged. - Clean room: own derivation (the conjugate form is elementary); no source beyond Candès, Wakin and Boyd, already cited. - Spec: no departure. Outputs change at rounding level on the fast path for a < ε, which is intended.
Tests.
- tests/python/test_nonconvex.py, new test_log_root_accuracy. Compare
against 80-digit decimal for the four reproducers above, plus a grid
a ∈ 10^[−300, 300], ε ∈ 10^[−300, 300], λ̃ = ρ ε² with ρ ∈ {1e-12, 0.5, 1}
(the convex regime). Assert relative error ≤ 4ε for the root on every
point where the exact output is nonzero and the distance to the
breakpoint is above 1e-12 relative. Assert prox([1e200]) ≈ 1e200
(rtol ε).
- The existing refined-grid minimiser test still passes.
- Add a finite-difference check of branch_deriv at ε = 1e-200, x = 1e-199.
As implemented.
- The usual path tests a single number, a ≤ 2⁴⁹⁹. The ε part of the range test is fixed at construction (eps_mid), and λ̃ ≤ 2⁹⁶⁰ always holds after A1's step range. The rescaled path is out of line (FP_COLD).
- mag, log_root and ElemCall::prox are forced inline (FP_INLINE). With the new root code mag had stopped being inlined, so tied and on went through memory, which cost 40% on the log penalty.
- For a < ε the root returns 0 before dividing when aε ≤ λ̃, since no positive root exists then. The tie test reads F(r) − F(0) in the normalised units, so it no longer underflows into a spurious tie.
- The continuity test is λ/ε ≤ ε, because ε² itself underflows.
Acceptance. uv run pytest -q tests/python/test_nonconvex.py passes.
The log case in CASES keeps passing. Speed: no measurable change, since
the fast path adds one range test and one select.
K4. tv1d_l2 derivatives are not scaled, so they fail outside about 1e±103¶
- Kind: correctness · Severity: medium · Target: 1.0.0 · Effort: M · Depends on: none
Problem. The prox normalises by a power of two, but setup, jvp and
vjp work on raw z (tv1d_l2.cpp:361-376, :410-421, :453-461).
s = ‖W D z‖ underflows in norm_wd (:364), and s·s·s under- or overflows
(:420, :460). Setup: fp.TV1DL2() with x = σ·[0, 1, 3, 2, 5] and
γ = 0.8σ; jvp with dx = e₀, jvp with dparams=[1], and vjp with g = e₀. z/σ
is correct at every σ:
| σ | jvp in dw | vjp weight gradient | Jacobian |
|---|---|---|---|
| 1e-110 | NaN | −inf | correct |
| 1e110 | NaN | 6.736e109, expected 3.368e109 (silently 2× off: the s³ term underflows) | correct |
| 1e-170 | 0 | 0 | identity ([1,0,0,0,0]) |
| 1e160 | NaN | NaN | identity ([1,0,0,0,0]) |
Design. H is invariant when z is replaced by t z and γ by t γ: c = γ/s
and p/s are scale-free. So setup normalises by the differences:
- dᵉ = z_{e+1} − z_e, in double. Let M = maxₑ wₑ|dₑ|. If M is not finite
(|z| near DBL_MAX), recompute dₑ = ½z_{e+1} − ½z_e and fold the ½ into t.
frexp(M, &k)gives t = 2^(−k) (or 2^(−k−1)). - d̂ₑ = t dₑ, ŝ = √(Σ aₑ d̂ₑ²) ∈ [0.5, √n]. If M = 0, return false as today. δₑ = d̂ₑ/ŝ (so |wₑ δₑ| ≤ 1), and c = γ/s = (γ t)/ŝ. If c is not finite, take the inside Jacobian: the piecewise mean, which is the limit of A⁻¹ as c → ∞.
- π = p/s, from πᵢ = Σₑ aₑ δₑ (e_{e+1} − e_e)ᵢ, and the factorisation
factor(1, c, inv, n)as today. - ŷ = A⁻¹(t·z_c) with z_c = z − (piece means). |z_c| ≤ n·M, so |t z_c| ≤ n. Let τ̂ = A⁻¹π (= A⁻¹p / s) and πᵀŷ = Σₑ aₑ δₑ (ŷ_{e+1} − ŷ_e), computed without cancellation as today. Since s·t = ŝ:
- pᵀy = s (πᵀŷ)/t;
- the Sherman–Morrison term c·(A⁻¹p)(A⁻¹p)ᵀv / (pᵀy) equals c·ŝ·τ̂(τ̂ᵀv) / (πᵀŷ).
So call apply_hinv(c, inv, τ̂, pty_hat, v, n) unchanged, with
pty_hat = (πᵀŷ) / ŝ.
5. JVP right-hand side, scale-free:
rᵢ = dxᵢ − dγ·πᵢ + Σₑ 2γ wₑ δₑ dwₑ·(1[i=e] − 1[i=e+1]) + γ·(Σₑ dwₑ wₑ δₑ²)·πᵢ.
This replaces :410-421, which uses Dz/s, Dz²/s³ and p.
6. VJP: q = H⁻¹ g, Σπq = Σᵢ πᵢ qᵢ. Then
acc[P] −= Σπq and
acc[e] −= γ wₑ δₑ·(2(q_{e+1} − q_e) − δₑ·Σπq).
This replaces :456-461.
Constraints.
- Determinism: yes.
- Allocation: none. The four n-double arrays suffice: inv, π (was p), ŷ (was
y), r or q. δ is recomputed from z on the fly or stored in the r buffer
before r is formed. The implementer should check the buffer
choreography; if a fifth array is needed, scratch_bytes grows from 4n to
5n doubles.
- ABI: unchanged. Clean room: own derivation (homogeneity). Spec: no
departure.
Tests.
- tests/python/test_tv1d_l2.py, new test_derivative_scale_invariance.
For σ ∈ {2⁻⁵⁶⁰, 2⁻³⁶⁵, 1, 2³⁶⁵, 2⁵³⁰}, x = σ·[0, 1, 3, 2, 5] and random
rows of length 50 with WE weights, γ = 0.8σ:
- jvp in x and in γ·σ equal the σ = 1 values (rtol 1e-12);
- jvp in w divided by σ equals the σ = 1 value (rtol 1e-12);
- the vjp weight gradient equals σ·g₁ (rtol 1e-12), so at σ = 1e110 it is
3.368e109;
- adjointness ⟨v, J u⟩ = ⟨Jᵀ v, u⟩ (rtol 1e-12).
- test_operators.py: tv1d_l2 and tv1d_l2_w in CASES keep passing.
As implemented.
- The four n-double arrays suffice. δ is recomputed from z as (h Δz)·(t/ŝ), with t/ŝ precomputed once, so each δ costs a multiply rather than a division.
- setup returns an explicit Kind:
- the identity, when z has no differences;
- the piecewise mean, when c overflows;
- the outside structure, otherwise.
- Measured at n = 10⁵, the jvp and vjp are within noise to 4% slower.
Acceptance. uv run pytest -q tests/python/test_tv1d_l2.py passes.
Speed: jvp and vjp of tv1d_l2 at n = 10⁵ are no more than 5% slower; one
extra pass for M.
K5. TreeL2 jvp and vjp: τ/s³ goes to inf for groups with ‖w_g‖ < 1e-103¶
- Kind: correctness · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: none (shares the scaled norm with K2 if K2 lands first)
Problem. group.cpp:308 and :351 compute tau / (s * s * s) * wt,
where s³ underflows for s < 5.6e-103 even though s² does not.
TreeL2([[0, 1]], [1e-111], 0) at x = 1e-110·[0.6, 0.8] gives jvp(e₀) =
[inf, inf] and a vjp of inf. GroupL2 on the same problem gives the correct
[0.936, 0.048].
Design. Rewrite the chain step with the unit vector û = w_g/s (with K2: û = (t_g w_g)/m_g):
- jvp: cc = τ/s, a = 1 − cc,
t_i <- a t_i + (cc * <û, t_g> - dtau) * û_i, thenw_i <- a w_i. - vjp: cc = τ/s, wu = <û_b, u_g>,
acc[1 + ei(g)] += -gamma * wu,acc[P] += -e * wu, andu_i <- a u_i + cc * wu * ûb_i, with ûb the group's saved input divided by its norm.
These are algebraically τ/s³·
Constraints.
- Determinism: yes. Allocation: none; ûb is formed on the fly from before.
- ABI: unchanged. Clean room: own algebra. Spec: no departure.
Tests.
- tests/python/test_operators.py, new test_tree_l2_small_scale. For
σ ∈ {1, 1e-110, 1e-150, 1e150}, x = σ·[0.6, 0.8], η = 0.1σ and l1 = 0.1σ
over the TREE fixture: jvp and vjp in x equal the σ = 1 values (rtol
1e-13), and the parameter gradients scale by σ⁰ for x and γ, and as
derived for η and l1, where dz/dη is scale-free. At σ = 1e-110,
jvp(e₀) = [0.936, 0.048].
- The tree_l2 and sparse_group CASES keep passing.
Acceptance. uv run pytest -q tests/python/test_operators.py -k tree.
Speed: unchanged; there is one multiply fewer.
K6. Lq: Newton stalls near the threshold (speed)¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: S · Depends on: none
Problem. In nonconvex.cpp:210-218 the general-q Newton stops when
|y_{n+1} − y| ≤ 4e-16·y. Near the branch start, where |x| is just above τ and
y ≈ β ≪ x, the rounding noise of h(y) = y + λ̃ q y^{q−1} − x is about ε·x/h′,
which exceeds 4e-16·y, so the iterates oscillate. The loop of root was
copied verbatim into a standalone C++ file, built with clang++ -O2 and run
with λ̃ = 1 and x = τ(1 + U(1e-9, 1e-3)) over 10⁵ draws:
| q | Mean iterations | Entries at the 100-iteration cap |
|---|---|---|
| 0.3 | 5.0 | 0 |
| 0.9 | 15.6 | 8.8% |
| 0.99 | 15.1 | 8.4% |
| 0.999 | 27.6 | 22% |
The library itself, fp.Lq(1.0, q).prox(x, threads=1) on 10⁶ entries (the
lq setups of kernel_throughput(), Apple M5), takes 380 ms for q = 0.9 and
703 ms for q = 0.999 near τ, against 53–68 ms far from it. The result is correct;
only time is lost. jvp and vjp pay the same again, because deriv1 and
deriv_piece recompute the root.
Design. Newton from the right on a convex increasing h decreases monotonically to the root (as the header already notes). So a step that fails to decrease y means the rounding floor has been reached. Replace the stopping test with:
double yn = y - h(y, lam, x) / dh(y, lam);
if (!(yn >= beta)) yn = 0.5 * (y + beta);
if (yn > x) yn = x;
if (!(yn < y)) break; // no decrease: at the rounding floor, keep y
const bool done = y - yn <= 4e-16 * y; // as before
y = yn;
if (done) break;
The first iteration starts at y = x with h(x) > 0, so yn < y. The closed forms (newton = 1) and the safeguard keep their behaviour. The final error stays at the noise floor ε·x/h′(y), the same bound as today.
Alternative rejected: a looser fixed tolerance (|step| ≤ 4ε·x). It is also correct, but it stops early far from the root's scale for q ≪ 1, while "no decrease" is parameter-free.
Constraints. - Determinism: yes. Allocation: none. ABI: unchanged. Clean room: no new source. Spec: no departure. - Outputs change at rounding level for near-threshold entries.
Tests.
- tests/python/test_nonconvex.py: test_global_minimiser_on_a_refined_grid
and test_closed_forms_match_newton still pass.
- New test_lq_near_threshold: q ∈ {0.3, 0.9, 0.99, 0.999},
x = τ(1 + 10^U(−12, −3)). The result satisfies h(z) = 0 to
|h| ≤ 8ε·x and z ≥ β.
- tests/c/test_core.cpp: the iteration count is not exposed through the
API, so the C test checks the result only.
Acceptance. Speed, from uv run python bench/bench_python.py on Apple
M5, kernel_throughput() lines:
| Line | Before | After |
|---|---|---|
lq q=0.9 near |
about 380 ns per entry | ≤ 90 |
lq q=0.999 near |
about 700 | ≤ 120 |
The lq ... far lines (about 53–68 ns) and every other line show no
regression.
As implemented. The stopping rule as designed, written as one test, plus a start closer to the root.
- Stopping. With d = y − yₙ, the loop stops when !(d > 4e-16·y) and takes yₙ only if d > 0. Written as the design's two tests, the far entries got 12% slower (50.5 → 57 ns); one compare is at parity.
- Start. The new stopping rule alone brought both near lines to 135 ns, missing the targets: from y = x the root, near β ≈ x/5, took 7.2 iterations on average and up to 9.
- h is convex, so it lies above its tangent at β, and h′(β) = 1 − q/2 for every λ (since β^(2−q) = 2λ(1−q)). The tangent's zero y₀ = β + (x − τ)/(1 − q/2) therefore bounds the root from above.
- root starts at min(x, y₀) for x < 2τ, where x − τ is exact, and at x beyond (from about 2τ on, y₀ > x). The factor 1/(1 − q/2) is a member, inv_slope.
- A y₀ below the root by rounding makes the first step go up; the descent continues from there. Without that case q = 0.999 near took 83 ns instead of 79.
- The closed forms (q = 1/2, 2/3) keep their single polish step.
- Measured. Through the C API, 10⁶ entries, threads 1, best of 7, six rounds alternating master's library and this branch's, Apple M5 Pro, ns per entry:
| Line | Before | After |
|---|---|---|
lq q=0.9 near |
543–546 | 48–49 |
lq q=0.999 near |
669–674 | 79–81 |
lq q=0.9 far |
51.1–51.9 | 52.4–53.1 |
lq q=0.999 far |
65.4–66.3 | 65.2–66.7 |
The kernel_throughput() lines through Python agree: 553 → 50 and 672 → 81 near, and 52.2 → 54.6 and 66.6 → 67.8 far.
- Open: lq q=0.9 far is 2.5% slower, above the 2% bound. A branch instead of fmin, the test on x alone and __builtin_expect each left it there. The far entries start at x as before, with the same iterations; what remains is the start's code ahead of the first pow.
- Bits. They change at rounding level for entries in (τ, 2τ), and for those whose iterates used to oscillate.
- Tests. test_lq_near_threshold: q ∈ {0.3, 0.9, 0.99, 0.999}, |x| = τ(1 + 10^U(−12, −3)) with both signs, λ = 0.7, γ = 1.3. Each result keeps the sign, has |z| ≥ β, and satisfies |h(|z|)| ≤ 8ε|x|.
K7. Hyperplane and half-space: Σ aᵢxᵢ overflows¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: M · Depends on: none
Problem. hyperplane.cpp:108-117 (unbounded, Lemma 6.26) and :123-131
(the pass sums c, s1) overflow when the entries are near DBL_MAX/n.
project_hyperplane([1.4e308, 1.4e308], a=[1, 1], b=0) returns [-inf, -inf]
and, with box ±1.5e308, [-1.5e308, -1.5e308]; [0, 0] is expected in both
cases. SumLargest calls plane_project with the same exposure.
Design. Make P_C homogeneous: P_{tC}(t x) = t P_C(x), where
tC = {aᵀz = t b, t lo ≤ z ≤ t hi}; a is already normalised to max |a| = 1.
In plane_project, which is the shared entry of HyperBox and SumLargest,
compute M = max(maxᵢ |xᵢ|, |b|, the finite |loᵢ|, |hiᵢ|). The x part comes
in the load pass that load_row already makes. The bound part is constant
per operator: compute it at construction as bmax. Then:
- Fast path: 2⁻⁹⁶⁰ ≤ M ≤ 2⁹⁶⁰ (n < 2⁶² keeps every sum below 2¹⁰²²), t = 1. This is unchanged.
- Otherwise
frexp(M, &k), t = 2^(−k). Scale the double row buffer x in place. Both callers own it as scratch: HyperBox'sxand SumLargest'sx. SetPlane::sc = tand b → t b. The bound accessors returnsc * lo[i]andsc * hi[i]; inf·t = inf, so the unbounded flag is unaffected. Solve, form ẑᵢ = clip(x̂ᵢ − μ̂ aᵢ, l̂ᵢ, ĥᵢ), and write zᵢ = ẑᵢ/t. The states (code) are scale-invariant. - To keep the hot loop unchanged, template
plane_solve,plane_startandplane_projectonbool Scaled. TheScaled = falseinstantiation is today's code. saved_mu(derivatives) uses the same scaling: μ = μ̂/t, with μ̂ from the scaled sums. μ enters only multiplied by daᵢ, so its overflow matters only for parameter derivatives.
Alternative rejected: compensated or two-pass summation. It does not prevent overflow of a sum whose true value exceeds DBL_MAX before the subtraction of b.
Constraints. - Determinism: yes. Allocation: none (in-place in the existing scratch). - ABI: unchanged. Clean room: Beck, Theorem 6.27, already cited; the scaling is elementary. Spec: no departure.
Tests.
- tests/python/test_planes.py, extend test_extreme_steps_and_scales:
t ∈ {2⁻¹⁰⁰⁰, 2⁹⁸⁰, 2¹⁰⁰⁰}. Hyperplane and half-space, with and without a
box, on the test_kkt_and_reference inputs scaled by t (x, b, lo and hi):
output equals t times the unscaled one (rtol 8ε). Add the reproducer as an
exact regression, [0, 0] both ways.
- sum_largest with x near 1e308 (k = 1, λ = 1) against the closed form.
As implemented.
- plane_scale returns the exponent k, and results go back with ldexp(·, k). Near DBL_MAX, k = 1024, so 2^k itself overflows.
- load_row reports max|x| during its existing pass. Each operator stores its largest finite bound (bmax) at construction.
- plane_solve, plane_start and plane_range are templated on Scaled. The false instantiation is today's code; the hyperplane, its box and the sums of the largest entries measured level with the old build.
Acceptance. uv run pytest -q tests/python/test_planes.py passes. Speed:
the hyperplane box and sum_largest k=10 lines of kernel_throughput()
in uv run python bench/bench_python.py show no regression. The fast path is the untouched Scaled = false
instantiation; there is one extra compare per row.
K8. Isotonic: block sums overflow; keep block means instead¶
- Kind: correctness (and a small speed gain) · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. isotonic.cpp:61-65 keeps block sums bs = Σ w x and compares
bs/bw on every push, which is two divisions per test. Sums overflow:
isotonic([1.5e308, 1.5e308, 0]) returns [inf, inf, inf]; expected
[1e308, 1e308, 1e308].
Design.
- Weights. At construction, store w/2^k with k from frexp(max w), so
the stored max is in [0.5, 1); remember k. Power-of-two division leaves
every weight ratio exact unless it is subnormal. Block weights are then
≤ n and never overflow.
- PAVA on means. Keep bm (the block mean, signed by sg) and bw (the block
weight). A push sets bw = wᵢ and bm = sg·xᵢ. Merge while
bm[top-2] > bm[top-1], with no division in the test:
W = bw2 + bw1; f = bw1 / W; // one division per merge
bm2 = bm2 + f * (bm1 - bm2) if both finite and |bm1 - bm2| <= DBL_MAX,
else (1 - f) * bm2 + f * bm1; // convex combination, no overflow
bw2 = W;
The update form is the more accurate and is used whenever bm1 − bm2 is
finite (one compare). The convex form covers opposite-signed extreme
means. W is never 0: weights with min/max below 1e-100 are rejected at
construction (decided 2026-10-22), as for the weighted ℓ1 ball, so every
stored weight is at least 2⁻³³⁴.
- Validation: make_isotonic returns FP_EPARAM when
min w / max w < 1e-100. The fp_op_isotonic comment in fastprox.h gains
"with min/max >= 1e-100", and the Python docstring says the same.
- Output: z = sg·bm per block.
- Derivatives (:89-120): with stored weights ŵ = w·2^(−k),
(Jv)ᵢ = Σ ŵⱼvⱼ/Ŵ is unchanged. The weight terms carry the scale:
jvp num += (x_i - z_s) * dp[i] * 2^-k before dividing by Ŵ, and vjp
acc[i] += (x_i - z_s) * m * 2^-k with m = G/Ŵ.
Constraints.
- Determinism: yes. Allocation: none; the bs array becomes bm, same size.
- ABI: unchanged. Clean room: PAVA is folklore, with no source copied
(Barlow et al. 1972 may be cited).
- Spec: a new rejection. Decisions row:
| 2026-10-22 | Weighted isotonic regression rejects weights with min/max below 1e-100 (FP_EPARAM), as the weighted l1 ball does: it keeps block means and weights scaled by their maximum, which must not underflow | src/ops/isotonic.cpp, fastprox.h |
- Outputs change at rounding level, since means replace sum/weight.
Tests.
- tests/python/test_operators.py: isotonic and isotonic_w in CASES
keep passing. New test_isotonic_extremes:
- the reproducer gives [1e308]×3 (rtol ε);
- [1e308, −1e308] (decreasing order, so they pool) gives [0, 0];
- weights [1e300, 1e-300, 1] (ratio 1e-600) and [1, 1e-101] raise
ValueError (FP_EPARAM), while [1, 1e-100] is accepted;
- parameter gradients at weight scale 2³⁰⁰ equal 2⁻³⁰⁰ times those at
scale 1 (rtol 1e-14).
As implemented. The pass on block means cost about 50% on a random walk, because its division sits on each merge's critical path. So the sum-based pass stays as the usual path; its weights are scaled by a power of two, so its results are unchanged, bit for bit. When a final block sum is not finite, the row is pooled again on means. Only overflowing rows pay for that.
Acceptance. Correctness: the tests above. Speed: the
isotonic random walk line of kernel_throughput() in
uv run python bench/bench_python.py is no slower than before. It is
expected to be 10–30% faster, since one division per merge replaces two per
test. Report the before and after numbers.
K9. TV1D constant_mean overflows¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. tv1d.cpp:231-241 forms a compensated sum of x, which
overflows for entries near DBL_MAX. tv1d([1.5e308, 1.6e308], 1e308) (the
constant-mean path) returns [nan, nan]; expected [1.55e308, 1.55e308].
Design. Keep today's compensated sum as the fast path. If hi + lo is
not finite, recompute around the midpoint of the range,
c = 0.5·min x + 0.5·max x (finite; the row's min and max are already known
from finite_range in prox, so pass them in). The terms are
(xᵢ − c)·2^(−j), with 2^j ≥ m the smallest power of two not below the row
length; each term is at most range/2 in magnitude before scaling, so the sum
cannot overflow. Then mean = c + (S/m)·2^j, with S the compensated sum of
the scaled terms. Also take this path when the fast sum is finite but
(hi + lo) / m is not.
Constraints. - Determinism: yes. Allocation: none. ABI: unchanged. - Clean room: no new source. Spec: no departure. The fast path is bit-identical.
Tests.
- tests/python/test_operators.py, new test_tv1d_constant_mean_extremes.
The reproducer gives [1.55e308, 1.55e308] (rtol 2ε). Also
[DBL_MAX, −DBL_MAX, DBL_MAX] with γ = DBL_MAX gives about DBL_MAX/3
(rtol 4ε).
As implemented. No change was needed: A1's step range makes this path unreachable. - The constant mean is taken when γw ≥ (n/2)·range. - The sum overflows only when n·|x| > DBL_MAX. - Any two distinct doubles near |x| differ by at least about |x|·2⁻⁵².
Both conditions together therefore need γ ≳ DBL_MAX·2⁻⁵³ ≈ 2e292, beyond 2⁹⁶⁰ ≈ 9.7e288.
Acceptance. uv run pytest -q tests/python/test_operators.py -k tv1d.
Speed: unchanged.
K10. Huber derivative: c² underflows¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. pointwise.cpp:147-152 forms c² = (δ + γλ)². It underflows for
c < 1.5e-154 and overflows for c > 1.3e154. Huber(1, 1e-200) at γ = 1e-300,
x = [0, 1e-201] gives jvp(dx = ones) = [nan, nan]: dg is NaN and NaN·0 = NaN
in run_jvp_. The vjp gives NaN gradients in γ, λ and δ. Expected:
jvp(ones) = [1, 1] to rounding, dz/dγ = [0, −0.1].
Design. On the inner branch, write q = x/c (|q| ≤ 1 there),
ρ_δ = δ/c and ρ_λ = (γλ)/c (both in [0, 1]):
e.j = rho_d, e.dg = -q * rho_d * lambda, e.pv[0] = -q * rho_d * gamma (dλ),
e.pv[1] = q * rho_l (dδ). No product grows past its factors. The outer
branch is unchanged.
Constraints. Determinism yes, allocation none, ABI unchanged, clean room none, spec no departure.
Tests.
- tests/python/test_operators.py, new test_huber_derivative_extremes.
For c ∈ {1e-200, 1e200}, both branches, the jvp in x, γ and θ against
central differences in log-space steps (h = 1e-6·scale), rtol 1e-6. The
reproducer gives [1, 1] and [0, −0.1].
Acceptance. The test passes and the huber CASES entry still passes.
K11. A degenerate bound lo = hi is treated as free¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. When loᵢ = hiᵢ the entry is fixed, so ∂z/∂x = 0 for the fixed θ. The "on a bound counts as interior" rule still selects the free piece at x = lo = hi:
Box([0, 0.5], [1, 0.5])at [0.3, 0.5] gives jvp(ones) = [1, 1], where [1, 0] is expected (pointwise.cpp:209).Hyperplane(ones(3), 1, lo=[0, .5, 0], hi=[1, .5, 1])at [.25, .5, .25] gives jvp(e₁) = [−1/3, 2/3, −1/3]. Both one-sided finite differences are [0, 0, 0] (hyperplane.cpp:127,:215).- L1Box with lo = hi has the same fault (
pointwise.cpp:363-367).
Design. The inside piece does not exist when l = h. Change the rule to: an entry with l = h is clipped, at the lower bound when x ≤ l and at the upper one when x > h. Then dz = dlo for x ≤ l and dz = dhi for x > h, which are the two pieces that meet in θ.
- Box
piece_of:{(x < l || (x == l && l == h)) ? 0 : (x > h ? 2 : 1), 1.0}.kink1keeps code 1 or 2. With l = h,deriv_at's sides b ∓ δ land on pieces 0 and 2, both clipped, so the override stays meaningful (dlo against dhi). - L1Box
piece_of:y < l || (y == l && l == h)gives piece 2. - HyperBox and SumLargest state assignment and solve loop:
lo = v < l || (v == l && l == h). In the branch-free loop (hyperplane.cpp:127) this is one more compare and an and. Inplane_solve's breakpoint collection (:159-165), skip entries with l = h, since they never change state. SumLargest's box [0, cap] with cap > 0 is never degenerate, so it is unaffected. - Documentation. The docs/api.md kink table's box row gets "(an entry with lo = hi is fixed: 0)", and the hyperplane row gets "free, unless loᵢ = hiᵢ (fixed)".
Constraints. - Determinism: yes. Allocation: none. ABI: unchanged. Clean room: none. - Spec: no departure; this is a clarification of the documented choice, which no longer selects a piece that does not exist.
Tests.
- tests/python/test_planes.py and tests/python/test_operators.py, new
test_degenerate_bounds, using the two reproducers and an L1Box one:
- the jvp in x equals the one-sided finite differences ([0, 0, 0] and
[1, 0]);
- with dparams, dz on the fixed entry equals dlo at x ≤ l and dhi at
x > h;
- adjointness holds (rtol 1e-14).
As implemented. The rule applies where it defines the derivative: the saved state (code), the box and L1Box pieces, and the solver's breakpoint list, which skips entries with lo = hi. The solver's branch-free state loop is unchanged. Its || form ran 15–100% slower on these unpredictable states, and an entry with lo = hi at an exact tie changes only that pass's Newton slope, not the root.
Acceptance. The tests pass and the hyperplane and box CASES still pass.
Speed: the hyperplane box line of kernel_throughput() in
uv run python bench/bench_python.py shows no regression.
K12. L2Ball jvp and vjp: fuse the passes and hoist the division (speed)¶
- Kind: speed · Severity: low · Target: 1.0.x · Effort: S · Depends on: K2
Problem. projections.cpp:260-292 makes three passes per row (norm, dot,
output) and divides each entry by s (x[i] / s) inside the output loop.
Design. Make one fused pass that accumulates Σxᵢ² and Σxᵢdxᵢ (or Σxᵢgᵢ)
together, in the fixed order and in independent lanes as in l1_threshold.
If Σxᵢ² fails K2's safe range, fall back to the scaled helper and redo the
dot on t·x. Then hoist inv = 1.0 / m (with K2, uᵢ = (t xᵢ)·inv) and run one
output pass. Results change at rounding level (x·(1/s) against x/s) and
stay deterministic.
Constraints. Determinism yes (fixed lanes), allocation none, ABI unchanged, spec no departure.
Tests. The l2_ball CASES (finite differences, adjointness, threading)
and the K2 homogeneity test.
Acceptance. In uv run python bench/bench_python.py on Apple M5, the
l2_ball jvp and l2_ball vjp lines of kernel_throughput() are at least
1.3× faster than before; report both numbers. l2_ball prox and the other
lines show no regression.
As implemented. As designed, and the prox shares the new sum of squares.
- One pass. with_norm_dot forms Σxᵢ² and Σxᵢvᵢ together in eight lanes in a fixed order. Out of the safe range it calls safe_norm_slow and redoes the dot on t·x in eight lanes, out of line. The output pass multiplies by inv = 1 / m. Inside the ball the jvp and the vjp copy with memcpy.
- The prox. The prox and value_row take Σxᵢ² from the same eight lanes (sumsq). The prox and its derivatives then agree on whether x is inside, as before, and the prox is faster too.
- Measured. One row of 10⁶, threads 1, Apple M5 Pro, ns per entry. Through the C API, best of 15, four rounds alternating master's library and this branch's; through Python, kernel_throughput(), best of three runs:
| Line | C API before → after | Python before → after |
|---|---|---|
l2_ball prox |
0.82 → 0.42 (1.9×) | 0.83 → 0.44 |
l2_ball jvp |
1.16 → 0.36 (3.2×) | 1.18 → 0.39 (3.0×) |
l2_ball vjp |
1.14 → 0.38 (3.0×) | 1.16 → 0.39 (3.0×) |
The other lines are within run-to-run noise. hyperplane box read 7.23 → 7.75 through Python, but in C it is 5.34 against 5.35: hyperplane.cpp is not changed.
- Bits. They change at rounding level: the lane order of both sums, and x·(1/m) against x/m. The radius gradient on the 10⁶ row moves by 6e-13 relative. Results are the same for any thread count.
- Tests. The l2_ball CASES and test_norm_scaling.py, whose homogeneity and subnormal tests run the jvp and vjp on the out-of-range path.
T. Image, graph and spectral operators¶
All measurements are from 2026-10-22 on an Apple M5 Pro (15 threads), at commit d67de78, through the Python package (uv run python) unless the C API is named. Line numbers refer to that commit.
Notation. - N is the number of entries of an image and k its number of axes (2 or 3). - eps = 2^-52. - In anisotropic TV, an edge e = (i, j = i + S_a) along axis a has the bound λ_e = γ w_a β_e. Its 1D dual is p_e = −(running sum of q_a along the fiber up to i). - The primal is P(z) = ½‖z − d‖² + Σ_e λ_e |z_j − z_i|.
Test image D(n, seed) (used throughout):
rng = np.random.default_rng(seed)
yy, xx = np.mgrid[0:n, 0:n] / n
D = ((xx - .5)**2 + (yy - .5)**2 < .1) + 0.5 * (xx > .7) + 0.1 * rng.standard_normal((n, n))
G(h, w) is the 4-connected grid as an edge list: all horizontal edges (i, i+1) in row-major order, then all vertical edges (i, i+w).
New benchmark sections. Each is a function added to bench/bench_python.py, called from __main__ after scaling(), timed with best(f, repeat=3, number=1). The speed items name them.
- B1 image_tv_derivatives(), titled "Isotropic TV derivatives, one image". For D(512, 0) at γ ∈ {0.1, 1}:
- TVIso(2).prox(x, γ, keep=True, tol=1e-6, max_iter=100000);
- then jvp(ctx, dx) and vjp(ctx, dx) with dx = default_rng(1).standard_normal(x.shape) and the default derivative tolerance.
- It prints the prox, jvp and vjp ms and the CG iterations (ctx.deriv), at threads 1 and 0.
- It also prints one line for D(2048, 11) at γ = 0.1, tol = 1e-4.
- B2 edge_tv_batches(), titled "Graph and tree TV, small batches".
- GraphTV(512², G(512, 512)) on a batch of 4 rows D(512, 4).ravel() + 0.01·default_rng(4).standard_normal((4, 512²)), at γ ∈ {0.1, 1}.
- TreeTV on n = 2·10⁵ nodes, with parent(i) = ⌊i·u_i⌋ for u = default_rng(6).random(n − 1). Batches of 4 and 16 rows of 0.1·cumsum(N(0,1)) + N(0,1) (default_rng(6)), γ = 1.
- Both at threads 1 and 0.
- B3 aniso_keep(), titled "Anisotropic TV, saved structure". TVAniso(2).prox(D(2048, 11), 0.1, tol=1e-4) with keep=False and keep=True, threads 0, printing both times and their difference.
T1. Anisotropic TV: flat edges at tiny bounds λ_e¶
- Kind: correctness · Severity: high · Target: 1.0.0 · Effort: M · Depends on: none
Problem. structure() (src/ops/tv_aniso.cpp:931) calls an edge flat when |p_e| < λ_e(1 − 10⁻⁹). The dual is computed as q = d + y − z, whose rounding is about eps·max|d| per entry. Once λ_e is below about 10⁷·eps·max|d|, edges whose optimal dual lies on the bound read as inside it, and their regions merge.
Reproducers:
- Tiny γ. 32² image default_rng(9).standard_normal((32, 32)), γ = 1e-8, tol = 1e-15, max_iter = 2·10⁵, threads 0. The smallest jump of x is 1.35e-4 > 2γ, so J = I exactly. Measured:
- ‖jvp(dx) − dx‖/‖dx‖ = 0.90; it is 0.97 at γ = 1e-12.
- 862 of the 1984 edges read as flat.
- Central finite differences at h = 1.35e-7 agree with I to 3e-10.
- On another seed it is 0.86.
- Tiny edge weights. 32² default_rng(10) N(0,1) image; β = 1 except half of the entries (mask default_rng(10).random((2, 32, 32)) < 0.5) set to 1e-6, so λ_e = 1e-8; γ = 0.01, tol = 1e-14. jvp against central finite differences is off by 0.66 (0.59 at β = 1e-8). The finite differences at h = 1e-7 and h = 1e-9 agree to 5e-8. With β = 1e-4 the error is 4.7e-8.
Tightening tol does not help: the gap meets any target after one or two iterations.
Design. Add a second, certified test that runs before the dual test. It uses the gap of the returned pair (z, q).
- Bound. P is 1-strongly convex, so for a feasible dual q with z = d − Σ_a q_a, ½‖z − z*‖₂² ≤ P(z) − P(z*) ≤ P(z) − D(q) = gap (Beck 2017, Lemma 12.7, already cited in the header). Hence ‖z − z*‖∞ ≤ ‖z − z*‖₂ ≤ ρ with ρ = √(2·gap).
- Certified jump. If |z_j − z_i| > 2ρ, then |z*_j − z*_i| ≥ |z_j − z_i| − 2ρ > 0. At the optimum z* − d + Σ_a D_aᵀ p*_a = 0 with p*_e ∈ λ_e ∂|·|(z*_j − z*_i), so every optimal dual has p*_e = λ_e·sign(z*_j − z*_i). The edge is a boundary of the solution's regions, and its sign is the sign of z's jump (z and z* differ by less than half the jump). Both claims hold whatever the computed p_e says.
- Floating point. Use τ = 2·√(2·gap⁺) + τ₀, with
- gap⁺ = max(gap, 0) + (M + 4)·eps·A,
- A = Σ_e λ_e |z_j − z_i| + Σ_p |z_p|·|d_p − z_p|,
- M = (k + 1)·N.
(M + 4)·eps·A bounds the error of the recursively summed gap; this is Higham's γ_M bound, and the gap is summed row by row in row_sum. The term
τ₀ = 64·eps·(max_p |d_p| + 2·Σ_a max_e λ_e)
covers the rounding of the z values and of the identity d − z = Σ q. It is the same constant edge_tv.hpp:189 uses for graph TV regions.
The main term is rigorous. τ₀ is a rounding allowance of the same standing as graph TV's. For the first reproducer (γ = 1e-8): A ≈ 3e-5 and τ ≈ 1.3e-8, against z jumps of at least 1.3e-4.
4. New edge rule in structure(), in this order:
1. A forced side from fp_workspace_set_branch (unchanged).
2. If λ_e > 0 and |z_j − z_i| > τ: boundary, with sign_e = sign(z_i − z_j). This is the zsign the sign pass already has. With edge weights, v.sign records the same sign.
3. Otherwise the existing dual test and the sign −sign(p_e).
Edges with λ_e = 0 keep today's rule. Compute A and max λ_e in one pass in structure(), using row_sum (fixed row order).
5. The gap inside structure(). prox_img calls structure() after adbpg/fdpg3 have set c.gap for the z they return (lines 299, 313); the exact paths have c.gap = 0. branch_image (line 1028) builds an ImageCall with no gap, so the saved structure must keep it. Its header grows from 8 to 16 bytes, (ncomp, gap), and View gets double *gap. This layout is internal, so there is no ABI effect. prox_img writes the gap; branch_image reads it into c.gap before calling structure().
As implemented:
- A and max|d| come from one serial pass in a fixed order, with d = z + Σ_a q_a. That is just as deterministic as row_sum and needs no partial-sum buffer in branch_image.
- The dual test runs first, and the jump is checked only where the dual would join the edge. Reading z along axis 0 for every edge cost 20%.
- The whole check is skipped when 2√(2·gap) is at least the range of z, so that no jump can pass it.
6. Optional follow-up (not needed for the fix). Apply the same rule in mark_edges/try_polish, which have (z, gz) at hand, so that polishing works at tiny λ_e. Today a polish there merges regions and is rejected by kPolishRatio.
Alternatives considered: - An absolute rounding allowance subtracted from λ_e in the dual test alone. Rejected: when λ_e is below the allowance, every edge becomes undecidable. - A threshold on |z_j − z_i| alone, without the dual. Rejected by the header's own measurement (errors of 0.07–0.19 on 128² images at a gap tolerance of 1e-8).
Isotropic TV is not affected. Its projection puts |u_p| on the sphere to relative rounding: on a 32² N(0,1) image at γ = 1e-14, the jvp is the identity to 3.4e-13.
Constraints.
- Determinism: the extra pass is row_sum in fixed order; the rule is local per edge.
- Allocation: none in prox/jvp/vjp; struct_bytes grows by 8 bytes per image.
- ABI: unchanged.
- Clean room: Beck 2017, Lemma 12.7; Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed., Section 4.2. Cite both in the kernel header next to the structure paragraph.
- No departure from docs/spec.md: this refines the 2026-10-16 region rule.
Tests (tests/python/test_operators.py or a new test_tv_structure.py):
1. The first reproducer at γ ∈ {1e-12, 1e-8}: ‖jvp(dx) − dx‖/‖dx‖ ≤ 1e-12, and the same for vjp. The same with 3D, 4×8×8.
2. The second reproducer with β_small ∈ {1e-6, 1e-8}:
- assert that central differences at h = 1e-7 and h = 1e-9 agree to 1e-6;
- assert ‖jvp − FD‖/‖FD‖ ≤ 1e-6.
3. In tests/c/test_core.cpp derivative_cases: TVAniso at γ = 1e-9 on a 16² randn image, with the identity Jacobian as reference.
4. The existing kink and branch tests must pass, because branch_image now reads the stored gap.
Acceptance.
- Tests 1–3 pass and every CASES entry for TVAniso passes unchanged.
- In B3 (uv run python bench/bench_python.py, section "Anisotropic TV, saved structure"), the keep=True time does not grow by more than 3%.
T2. Isotropic TV: threaded derivative solves¶
- Kind: speed · Severity: high · Target: 1.0.x · Effort: L · Depends on: T3
Problem. image_jvp_driver and image_vjp_driver (src/core/driver.hpp:672, :730) parallelize only across images (B >= 2 ? nt : 1). jvp_image/vjp_image receive no thread count, so a single image's conjugate gradients (src/ops/tv_iso.cpp:677) run on one thread. Measured on one image, with keep=True, tol = 1e-6 and the default derivative tolerance:
| Image | γ | prox (threads 0) | jvp |
|---|---|---|---|
| D(512, 0) | 0.1 | 119 ms | 409 ms (170 CG iterations), the same at threads 1 and 0 |
| D(512, 0) | 1 | 684 ms | 1413 ms (649 iterations) |
| D(2048, 11) | 0.1, tol 1e-4 | 154 ms | 1605 ms (vjp 1616 ms) |
Through Python (seed 0, tol 1e-6) jvp takes 403 ms at threads 1 and at threads 0. On 512² each CG iteration costs about 2.4 ms serially.
Design.
- Plumbing.
DerivCtl(src/core/op.hpp:89, internal) gainsint nt = 1;.- The jvp driver becomes the prox driver's shape:
outer = B >= 2 && B >= nt.- Outer: images in parallel with
dc.nt = 1. - Otherwise: images one after another,
dc.nt = ws.nthreads, set 0.
- Outer: images in parallel with
- The vjp driver keeps its fixed blocks
nb(reduction_blocks). When B < nt it runs the blocks in block order on the calling thread, each image withdc.nt = ws.nthreads; otherwise as now. The accumulators per block do not change, so the cross-image sums keep their order. - Aniso, graph and tree TV ignore
dc.nt. - Region membership (built once, in
save(),tv_iso.cpp:474). After the labels, a counting sort produces CSR arrays: rstart[nc + 1](int32);members[N](int32), each region's pixels in increasing index.
struct_bytes grows by r8(4N) + r8(4(N + 1)). This is serial O(N) and runs once per prox.
3. Pixel-space passes. These are grad, hess, sub_gradT, the gather pix[p] = cv[label[p]], and the right-hand-side loops in jvp_img. Each runs over rows with parallel_for(dc.nt, rows, max(1, kChunk / n), …), in the row form of T3. They have no reductions, and sub_gradT writes only its own row, so there are no races.
4. Region sums M^T y (apply's scatter out[label[p]] -= pix[p], and the scatter of the right-hand side into cb). Use a segmented sum over members in fixed chunks of kSeg = 4096 positions:
1. Chunk pass (parallel over chunks). A region whose members all lie in one chunk is summed directly, in member order. That is the order of today's serial loop, so its bits do not change. A region cut by chunk boundaries is summed as one piece per chunk, each in member order, into piece[2·nchunk]: the piece that starts the chunk and the piece that ends it. A chunk covered by a single region has one piece.
2. Serial pass over the boundaries. Each crossing region's total is its pieces added in chunk order. There are at most nchunk crossing regions.
Everything is fixed by (N, labels), so the result does not depend on nt.
5. Region-space vector updates and dot products (pq, rzn, rn, bn, rz): blocks of kRegBlk = 4096 regions. Element updates run in parallel. Dot products write one partial per block into rpart[ceil(nc/4096)], then sum it in block order.
6. vjp parameter sums (tv_iso.cpp:777 onward: ‖·‖ terms per axis, ggam): row_sum-style partials per row (the part buffer of rows doubles already in image_bytes), summed in row order. acc[k + p] (pixel weights) is per pixel and has no reduction.
7. jacobi stays serial. It runs once per solve and costs about one apply; after T2 and T3 that is under 3% of a 170-iteration solve. Its scatter into up to k + 1 regions per pixel would need a pull formulation to be race-free.
Work memory: image_bytes (tv_iso.cpp:155) adds (3·ceil(N/4096) + 8) doubles for piece and rpart.
Bits do change against today: crossing regions are summed in pieces, and dot products are blocked. They are identical for every thread count.
Alternatives considered:
- Atomic adds into regions. Rejected: the order depends on scheduling.
- Per-thread region accumulators reduced at the end. Rejected: the partition depends on nt, and it costs nc·nt memory.
- Hybrid threading of B ≥ 2 images (nt/B threads each). Rejected for now: the pool runs nested callers serially (parallel.cpp).
Constraints.
- Determinism: chunks, region blocks and row partials are fixed by N and the saved structure, never by nt.
- Allocation: build_workspace sizes struct_bytes + 8N + 8 bytes per image and image_bytes + 8·(3·ceil(N/4096) + 8) bytes per set. No allocation in jvp/vjp.
- ABI: unchanged (DerivCtl is internal).
- Clean room: no new sources (CG is textbook: Saad, Iterative Methods for Sparse Linear Systems, Algorithm 9.1, preconditioned CG).
- No departure from the spec.
Tests.
- tests/python/test_operators.py threading test, extended to TVIso derivatives: 2 × 128² and 1 × 256² images, γ ∈ {0.03, 0.3}, tol 1e-8. Assert jvp, vjp, ctx.deriv (residual, iters) and grads are bit-identical across threads ∈ {1, 2, 3, 7, 15}.
- tests/c/test_core.cpp allocation counter: one fp_jvp and one fp_vjp of TVIso at threads = 4 allocate nothing.
- The existing finite-difference and adjointness CASES pass.
Acceptance. Add B1 with this change. uv run python bench/bench_python.py, section "Isotropic TV derivatives, one image", threads 0, with T3:
- D(512, 0) jvp at γ = 0.1: 409 → ≤ 80 ms;
- at γ = 1: 1413 → ≤ 250 ms;
- D(2048, 11) jvp: 1605 → ≤ 300 ms.
vjp follows the same targets. Threads 1 is at most 3% slower than T3 alone, and the existing sections do not regress.
As implemented. As designed, with three changes.
- When an image takes all the threads. B >= nt alone made a batch of small images slower: 2 × 128² went 12.7 → 19.1 ms at 15 threads, since a 128² image is one chunk and the two images then ran one after the other. derivs_inside in driver.hpp runs a batch one image at a time only when an image's chunks of kChunk entries give at least twice the width of the images in parallel (min(threads, ⌈N/kChunk⌉) ≥ 2 min(threads, B)); a single image always does. Operators say whether they thread their derivatives with fp_op::threads_derivatives(), true only for TVIso, so anisotropic, graph and tree TV keep their images in parallel. The vjp's fixed blocks and accumulators are the same in both modes, so the choice does not change the bits.
- Sums. The vjp's parameter sums and the CG dot products use fixed blocks: of kChunk / n rows for the former (into part), of kRegBlk = 4096 regions for the latter (into rpart, two sums per block, as rzn and rn are computed together). The region sums run over chunks of kSeg = 4096 positions of members; a region inside one chunk is summed in one loop in pixel order, as the old scatter did, and one cut by chunk boundaries as a piece per chunk added in chunk order. The work memory grows by 4⌈N/4096⌉ doubles.
- Bits. An image of at most one chunk, one row block and one region block (64² and below) keeps the bits of the old serial code, so the DERIVATIVE_BITS recorded by T3 on D(64, 0) still hold and were not re-recorded. Larger images change within the solve's tolerance: D(512, 0) relative 7e-14 (γ = 0.1, 181 CG iterations as before), 4e-5 for the jvp at γ = 1 (728 → 730), 256² with pixel weights at tol 1e-8 3e-6 (1106 → 1103).
- Also. One thread is 22–37% faster. apply fuses the gradient and the Hessian into one row pass, and the region sums read each region's pixels in turn instead of scattering into out; which of the two gains more was not measured.
- Tests. test_derivative_threads in tests/python/test_tv_iso.py: jvp (with dγ and dparams), vjp, the gradients and ctx.deriv at threads 1, 2, 3, 7, 15 on 2 × 256² (images in parallel at 2 and 3 threads, one at a time at 7 and 15), 256² at γ = 0.03 and 0.3, 256² with pixel weights, and a 3D 12 × 48 × 64. It fails on block orders that depend on the thread count, in the dot products, in the row sums and in the region pieces (checked by mutation). test_no_allocation in tests/c/test_core.cpp runs jvp and vjp at 4 threads on 256 × 192 and 2 × 256², with and without pixel weights. A C harness ran ASan over shapes whose rows, chunks and blocks end unevenly (1 × 4099, 4099 × 1, 65 × 67, 3 × 64 × 129, 3D) and TSan on 512 × 384, 2 × 300² and 24 × 64², checking threads 3, 8 and 16 against 1: no reports, identical bits.
- Measured. B1, Apple M5 Pro (load 4–6 throughout, so the 15-thread figures vary), two runs alternating master and this branch, ms:
| master | T2 | |
|---|---|---|
| D(512, 0) γ = 0.1 jvp / vjp, threads 1 | 278 / 279, 277 / 276 | 216 / 217, 218 / 230 |
| D(512, 0) γ = 0.1 jvp / vjp, threads 0 | 417 / 299, 460 / 357 | 79 / 88, 76 / 84 |
| D(512, 0) γ = 1 jvp / vjp, threads 1 | 1007 / 1001, 987 / 999 | 627 / 652, 641 / 637 |
| D(512, 0) γ = 1 jvp / vjp, threads 0 | 1034 / 980, 1006 / 997 | 275 / 289, 275 / 269 |
| D(2048, 11) jvp / vjp, threads 1 | 996 / 1031, 1071 / 1089 | 706 / 722, 733 / 745 |
| D(2048, 11) jvp / vjp, threads 0 | 1092 / 1086, 1071 / 1087 | 251 / 212, 234 / 237 |
Master's threads-0 lines are serial, so they are its threads-1 lines plus the load. An earlier run under less load gave T2 at threads 0 40 / 42, 126 / 124 and 100 / 102 ms, which meets the targets (≤ 80, ≤ 250, ≤ 300); the loaded runs above miss the γ = 1 target by 10% and the vjp at γ = 0.1 by 8%. Batches at threads 0 (jvp, ms): 4 × 256² 93, 90 → 65, 78; 16 × 64² 8.9, 8.1 → 6.5, 8.8; 2 × 128² 14.2, 12.9 → 10.6, 11.4; 32 × 48 × 48 (3D) 7.8, 6.8 → 4.9, 4.7. Single images of 64² and 128² run on one thread as before; measured alone, 64² is 2.67 → 2.34 ms at every thread count.
T3. Isotropic TV: integer divisions in the derivative loops¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: S · Depends on: none
Problem. Sys::grad and Sys::sub_gradT (tv_iso.cpp:587, :596) call g.has_next(p, a) and g.coord(p, a), which compute (p / S_a) % L_a, for every pixel and axis in every CG iteration. The same pattern appears in:
- jacobi (:626);
- the right-hand side of jvp_img (:734 onward);
- the per-axis loop of vjp_img;
- save().
Measured through the C API: a 512² disk-and-step image with uniform noise of width 0.35 (the D layout, noise from an LCG s ← 1103515245·s + 12345, seed 1), tol = 1e-6, threads 0, so the jvp itself is serial. Replacing grad and sub_gradT with the row loops below took the jvp from 413 to 266 ms at γ = 0.1, and from 1357 to 835 ms at γ = 1. The output checksum is bit-identical and the CG iteration counts are the same. In isolation, G plus Gᵀ on 512² went from 1.02 to 0.22 ms per application.
Design. Iterate rows r of length n = L[k−1] with base b = r·n.
- Gradient y_a:
- Along the last axis: y[b+j] = w·(x[b+j+1] − x[b+j]) for j < n − 1, and y[b+n−1] = 0.
- Along an axis a < k − 1: test g.has_next(b, a) once per row, then y[b+j] = w_a·(x[b+j+S_a] − x[b+j]) for all j, or 0 for the whole row.
- Adjoint x −= Gᵀh:
- Along the last axis: x[b] += w·h[b] and x[b+j] += w·(h[b+j] − h[b+j−1]) for j ≥ 1.
- Along a < k − 1: test g.coord(b, a) > 0 once per row, then x[b+j] += w_a·(h[b+j] − h[b+j−S_a]), or x[b+j] += w_a·h[b+j] on the first row along a.
The arithmetic per entry is unchanged, so results are bit-identical. jacobi, jvp_img, vjp_img and save() take the same form, with has_next per row precomputed into bool nxt[3] as grad_rows already does.
Constraints. - Determinism and outputs: bit-identical. - Allocation: none. - ABI: unchanged. - Clean room: n/a. - Spec: no departure.
Tests. The existing TVIso CASES pass. A new assertion checks that jvp/vjp outputs and ctx.deriv equal values recorded before the change on a fixed 64² image: checksums stored in the test, bit-exact.
Acceptance. B1 (added by T2, or with this change if it lands first), section "Isotropic TV derivatives, one image", threads 1: - D(512, 0) jvp at γ = 0.1: 409 → ≤ 280 ms; - at γ = 1: 1413 → ≤ 900 ms.
As implemented. As designed. B1 lands with it, as image_tv_derivatives() in bench/bench_python.py.
- Loops. grad and sub_gradT run by rows, through add_diff (x += c(h_p − h_{p−e_a}), also the dparams term of jvp_img's right-hand side). The per-axis sums of vjp_img run by rows in entry order. jacobi and save() keep their per-entry bodies and carry the column j as a wrapping counter, with row_next filling nxt[a] at the start of each row. kink_image keeps its per-entry form: it runs once per call.
- Bits. Identical to master. Checked by hashing, before and after:
- jvp (plain, with dγ, with dparams), vjp, the parameter gradients and ctx.deriv;
- on six setups: 2D, 2D with axis weights, 2D with pixel weights, 3D, 3D with a zero axis weight, and a batch of two.
- Measured. B1, Apple M5 Pro, two runs alternating master and this branch, ms. CG iteration counts are unchanged, and the prox lines are not changed.
| Setup | jvp before → after | vjp before → after |
|---|---|---|
| D(512, 0), γ = 0.1, threads 1 (174 CG iterations) | 395–406 → 262–265 | 397–407 → 263–266 |
| D(512, 0), γ = 1, threads 1 (731) | 1503–1524 → 919–930 | 1494–1523 → 918–926 |
| D(2048, 11), γ = 0.1, tol 1e-4, threads 1 (34) | 1368–1396 → 936–939 | 1388–1414 → 951–957 |
Threads 0 reads the same, because the derivative is serial until T2.
- γ = 0.1 meets its target.
- γ = 1 is at 919–930 against ≤ 900. This baseline runs 731 CG iterations, against the 649 measured when the audit was written. The ratio, 0.61, is below the target's 900/1413 = 0.64.
- Tests. test_derivative_bits in test_tv_iso.py holds the checksums of jvp, vjp, grads and ctx.deriv on D(64, 0), with and without weights, recorded on master and bit-exact.
T4. Graph and tree TV: small batches run serially¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: S · Depends on: none
Problem. image_prox_driver (src/core/driver.hpp:569) runs images in parallel only when B >= 2 && B >= nt. Otherwise it runs them one by one with nt inner threads. Graph and tree TV never use the inner threads, so a batch smaller than the thread count runs on one thread. Measured with the B2 setup:
- GraphTV, batch of 4 on G(512, 512), γ = 0.1: 3278 ms at threads 0 and 3271 ms at threads 1. At γ = 1: 16.8 s and 16.8 s.
- TreeTV, batch of 4: 40.1 and 41.2 ms. A batch of 16 is parallel: 82 and 166 ms.
Design.
- Add virtual bool splits_image() const noexcept { return false; } to fp_op (src/core/op.hpp, internal). TVAniso and TVIso override it to return true, since they use ImageCall::nt.
- The existing splits_rows() is not reused: it serves the device check (api.cpp:1379), and TVIso does not set it.
- In the driver: outer = B >= 2 && (B >= nt || !op.splits_image()).
- ws.image_sets = min(B, nthreads) (api.cpp:186) already provides one working set per concurrent image, and the per-thread scratch is sized for nthreads.
Constraints. - Determinism: images are independent, and gaps and iters are written per image. - Allocation: none new. - ABI: unchanged. - Spec: no departure.
Tests. test_operators.py threading test for GraphTV and TreeTV with B = 3 at threads ∈ {1, 2, 8}: outputs, info.gap, info.iters and the dual are bit-identical.
Acceptance. Add B2 with this change. uv run python bench/bench_python.py, section "Graph and tree TV, small batches", threads 0:
- GraphTV batch of 4 at γ = 0.1: 3278 → ≤ 1000 ms;
- TreeTV batch of 4: 40 → ≤ 15 ms;
- the threads 1 lines and the TreeTV batch of 16 do not change.
As implemented. As designed. splits_image() is set by TVAniso and TVIso. Both image drivers, the prox's and fp_workspace_refine's, use outer = B >= 2 && (B >= nt || !op.splits_image()). B2 lands with it, as edge_tv_batches().
- Measured. B2, Apple M5 Pro, one run each of master and this branch, ms:
| Line | threads 1, before → after | threads 0, before → after | Target |
|---|---|---|---|
| GraphTV, batch of 4, γ = 0.1 | 2973 → 3012 | 2979 → 939 | ≤ 1000 ✓ |
| GraphTV, batch of 4, γ = 1 | 18607 → 18765 | 18679 → 5354 | — |
| TreeTV, batch of 4 | 39.4 → 41.1 | 39.4 → 12.3 | ≤ 15 ✓ |
| TreeTV, batch of 16 | 137 → 143 | 19.3 → 19.7 | unchanged |
The threads 1 lines and the batch of 16 run the same code as before; their differences are run-to-run noise on a loaded machine. Four images on 15 threads give 3.2× for graph TV, not 4×, since the images take different numbers of max-flow solves.
- Tests.
- test_small_batches_threads in test_graph_tv.py (GraphTV and TreeTV, B = 3, threads 1, 2, 8): z, info.gap, info.iters and the dual are bit-identical.
- The same check is in test_tree_tv in test_core.cpp, so that ThreadSanitizer runs the new parallel path (it passes, as do ASan and the fuzzer).
T5. Image TV: overflow of ‖x‖² and of the gap (power-of-two rescaling)¶
- Kind: correctness · Severity: medium · Target: 1.0.0 · Effort: M · Depends on: none
Problem. The stopping target tol·‖d‖²/2 and the gap terms z·(d − z) overflow once max|x| ≳ 1.3e154 (tv_iso.cpp:402, tv_aniso.cpp:304-310).
Setup: x = default_rng(2).standard_normal((16, 16)) (3D: (4, 8, 8)), reference z_ref = solve(x, 0.5, tol=1e-12, max_iter=100000). Then solve(c·x, 0.5·c) at c = 1e155 should equal c·z_ref.
- Isotropic TV (2D and 3D). The cold start has gp = ∞ against a target of ∞, so the loop never runs (tv_iso.cpp:413). It returns z = x with iters = 0, converged = True and FP_OK: relative error 1.69 (2D) and 2.3 (3D).
- Conjugate prox. prox_conj(x, σ = 1e-156) returns 0 instead of the projection (max difference 2.8 against σ = 1e-3).
- 3D anisotropic (fdpg3, :537). It returns the 1D prox along the last axis only: relative error 2, iters = 0, FP_ENOTCONV.
- 2D anisotropic. The answer is right, but the gap is NaN, so it runs all of max_iter (100000 iterations).
Design. g is positively 1-homogeneous, so for c > 0, prox_{γg}(c·d) = c·prox_{(γ/c)g}(d). The dual scales by c and the gap by c². With c a power of two, every scaling is exact.
- Helper in
src/ops/image_util.hpp:It sets m = max_p |d_p|. If m == 0, it returns 0. Otherwise it sets e = ilogb(m) and returns 0 if |e| ≤ 64, or if e < 0 and γ > ldexp(DBL_MAX, e); else it returns e.// Exponent e for solving on 2^-e d with step 2^-e gamma (exact scalings); // 0 when max|d| is within [2^-64, 2^65), when d == 0, or when 2^-e gamma // would overflow. inline int rescale_exponent(const double *d, int64_t N, double gamma) noexcept; - In TVIso::prox_img and TVAniso::prox_img, after the NaN/Inf scan and before any branch on the number of active axes, when e ≠ 0:
- Scale d and the warm dual q by 2^-e, and set γ̂ = ldexp(c.gamma, −e). Run the existing code on a local copy of
ImageCallwithgamma = γ̂. The target becomes tol·‖d̂‖²/2 with ‖d̂‖∞ ∈ [1, 2), so no square overflows, and the stopping decisionconvergedis made on the scaled gap. - Afterwards, z and q are multiplied by 2^e, and
c.gap = ldexp(ĝap, 2e)(it may be +inf;convergedstays as decided). - Saved structure: TVAniso's (labels, 1/|C|, signs) is scale-free; T1's stored gap is ĝap·2^{2e}. TVIso's
save()runs on the scaled call, then multiplies the saved z, u and mu by 2^e (relis scale-free). - When e = 0 nothing changes, so the output is bit-identical to today for every image with max|x| in [2^-64, 2^65).
- As implemented:
- Isotropic TV keeps its saved z, u and μ at the solved scale and stores e in the saved header, which grows from 24 to 32 bytes. Scaling them back, as point 2.3 says, fails past 2⁵¹², where the derivative code's |(W D z)_p|² overflows; at 2⁷⁰⁰ the jvp was off by 2.3.
jvp_imgandvjp_imginstead run with γ̂ = 2^-e γ. Since z = 2^e ẑ(2^-e x, 2^-e γ, θ), the derivatives in x and γ are ẑ's, and θ's carry 2^e: tangents in θ are multiplied by 2^e in the jvp, and θ's gradients in the vjp. sc = 1 when e = 0, so in-range images are bit-identical. - No rescaling when no axis is active: z = x exactly there. Also none when 2^-e γ w_a would underflow to 0 on an active axis.
- The tests add negative exponents (tiny images underflowed the same way) and a jvp/vjp check at 2⁷⁰⁰. Test 3 asserts a gap that is not NaN; it is +inf at 1e300, as the contract allows.
- The fuzz inputs are scaled only for image TV in float64: 2^600 does not fit in float32, and the row operators' overflows (K2–K9) are separate items.
New contract text in fastprox.h (iterative image operators) and docs/api.md:
"Inputs are rescaled internally by a power of two, so the result does not depend on the scale of x: for any c = 2^k, prox(c·x, c·γ) = c·prox(x, γ) up to rounding. The reported gap may be +inf when it exceeds the largest double; convergence is decided on the scaled problem."
Alternatives considered: - Scaling only the target and the gap terms. Rejected: it leaves the iterates near DBL_MAX, and the dual test of T1 depends on scale. - Rescaling by max|d| itself. Rejected: it is not exact, and it would change the bits for in-range data.
Constraints. - Determinism: a per-image scalar decision. - Allocation: none. - ABI: unchanged. - Clean room: Bauschke and Combettes, Convex Analysis and Monotone Operator Theory, Proposition 24.8(iii) (prox of a scaled function), plus homogeneity. - Spec: no departure.
Tests (tests/python/test_tv_scaling.py):
1. For TVAniso(2), TVAniso(3), TVIso(2), TVIso(3) on the setup above, at γ = 0.5, tol 1e-12, and each k ∈ {300, 600, 1000} with x·2^k and γ·2^k:
- ‖z/2^k − z_ref‖∞ ≤ 1e-10;
- converged is True;
- outputs divided by 2^k are bit-identical between k = 600 and k = 1000.
2. Each operator gives prox_conj(x, 1e-156) within 1e-10 of prox_conj(x, 1e-3).
3. TVAniso(2) at x·1e300 with γ = 1e300 finishes in under 100 iterations with a non-NaN gap.
4. The fuzz harness (fuzz/fuzz_ops.cpp) adds image-TV inputs scaled by 2^±(600–1000).
Acceptance. Tests 1–3 pass, and all existing tests are bit-identical (e = 0 path).
T6. Graph TV: dual and gap lost to cancellation at large γ·w¶
- Kind: correctness · Severity: low-medium · Target: 1.0.0 · Effort: M · Depends on: none
Problem. The flow of a live edge is read as cap − mf.residual(arc) (src/ops/graph_tv.cpp:397). Once cap = γw_e far exceeds the flow, the residual has already absorbed it, because the engine stores r_uv = cap − f and loses f below ulp(cap).
Setup: GraphTV(256, G(16, 16)) with x = default_rng(5).standard_normal(256), tol = 0. z is exact everywhere, but the returned dual is wrong:
| Setting | ‖x − z − q‖∞ | gap | converged |
|---|---|---|---|
| γ = 1e10 | 2.7e-6 | 7e-11 | True |
| γ = 1e15 | — | 0.33 | True |
| γ ≥ 1e17 | 2.8 | 121 | True |
| γ = 1e300, w = 1e10 (cap overflows) | — | NaN | True; dual non-finite |
The saved flow, which decides kinks, inherits the same error.
Design.
1. Exact net flow in the engine.
- maxflow::EIBFS (src/maxflow/eibfs.hpp) gets a per-arc array T f_[2m] of net flow. It is sized in bytes() (+ rnd(sizeof(T)·2m)), zeroed in init_edge, and updated in augment (:220) next to every residual update: f_[a] += delta; f_[sister[a]] -= delta. It is exposed as T flow(Index a).
- set_edge leaves f_ unchanged; deleted edges use esgn.
- maxflow::MaxFlow (BK) gets the same array, so the -DFASTPROX_GRAPH_BK build certifies the same way.
- The certificate flow becomes esgn[e] ? esgn[e]·cap : mf.flow(g.arc[e]). Its error is a few eps·|f| instead of eps·cap.
2. Infinite capacities.
- In edge_tv::Base::certificate (edge_tv.hpp:146), the edge term is cap·|Δz| − f·Δz with Δz = z_u − z_v. Skip it when Δz == 0 (it is 0, and inf·0 would be NaN). It is ≥ 0 otherwise.
- In structure() (:173), accumulate deg only over finite caps. An infinite-capacity edge is never cut, so it adds nothing to the region values whose rounding τ bounds.
3. Tree TV uses its subtree sums (exact) and gets item 2 only.
Alternatives considered: - Clamping capacities at a bound on the flow. Rejected: a sound bound must cover the flow accumulated over all rounds, including the terminal changes from deleted edges, and no simple bound was found. - Recovering f as (r_vu − r_uv)/2. Rejected: the same cancellation.
Constraints.
- Determinism: unchanged (same augmentations).
- Allocation: image_bytes grows by 16m bytes per working set (8.4 MB on G(512, 512), against about 19 MB today); build_workspace sizes it through MF::bytes.
- ABI: unchanged.
- Clean room: no new sources; the EIBFS and BK papers are already cited.
- Spec: no departure.
Tests (tests/python/test_operators.py, GraphTV):
- The setup above, at γ ∈ {1e10, 1e17, 1e20}: ‖x − z − info.dual[0]‖∞ ≤ 1e-12·max|x| and info.gap ≤ 1e-20·‖x‖².
- Weights 1e10 at γ = 1e300: the gap is finite (or 0), the dual is finite, and jvp(ones) = ones.
- The C engine tests in tests/c/test_core.cpp: the flow equals cap − residual on random graphs with caps ≤ 1e3 (to 1e-12).
As implemented.
- Pushed amounts instead of a net-flow array. Each engine keeps p[a], the total pushed along arc a, with one store per path arc (the arc whose residual drops). flow(a) = p[a] − p[sister a]. The rounding is eps·Σ|δ|, the same as a signed net-flow array that would need two stores.
- Tracking is opt-in. It is enabled by track_flow() before init_edge, and graph TV enables it only when γ·max w > 64·(max x − min x). Below that, cap − residual is accurate to 64ε of the value range and is kept.
- Why opt-in. With tracking always on, GraphTV on a 512² grid at γ = 1 took 8% longer, because zeroing and updating the 2m extra doubles adds about 8 MB of memory traffic. Gated, the 256² and 512² grids at γ = 0.1 and 1, and the BK engine in bench/bench_maxflow.cpp, are within noise of the old code.
- Memory. bytes() still grows by 2m values, as designed.
- Design item 2 is unreachable after A1. The step range bounds γ·max w by 2⁹⁶⁰, so capacities are finite and the reproducer's γ = 1e300 with w = 1e10 is FP_EPARAM. The edge term cap·|Δz| is 0 at Δz = 0 and can at worst overflow to +inf (never NaN), and the degree sums of structure() stay finite. The code is unchanged. In its place, test_dual_at_the_step_bound runs at γ·w = 2⁹⁵⁹.
- Engine tests. test_maxflow.cpp (both engines) checks the net flow against cap − residual on every fresh and reused problem, the latter across set_edge. test_large_caps checks conservation at every node with capacities ×1e17, ×1e20 and ×1e280; it fails in all 12 cases with cap − residual.
Acceptance.
- The tests pass.
- In B2 (uv run python bench/bench_python.py, section "Graph and tree TV, small batches"), the GraphTV threads 1 lines do not slow down by more than 5%. Today the γ = 1 batch of 4 takes 16.8 s, and one image at γ = 1 about 3.9 s. The BK build is held to the same bound.
T7. Spectral derivatives: near-ties chained into one wide cluster¶
- Kind: correctness · Severity: low-medium · Target: 1.0.0 · Effort: S · Depends on: none
Problem. spectral_driver groups sorted values into clusters by comparing adjacent gaps to band = eps^(1/3)·max|s| (src/core/spectral.hpp:130). It then gives every pair in a cluster the averaged limit ½(λ_i + λ_j) (:158). A run of closely spaced values chains into one cluster that is much wider than the band.
Setup: NuclearNorm(λ), k = 200; U and V are the Q factors of np.linalg.qr of two default_rng(7).standard_normal((200, 200)) matrices; s_i = 1 + 5e-6·i, sorted descending (spacing under the band of 6.06e-6, total width 1e-3); λ = 1 + 5e-4; prox_from(U, s, V.T, keep=True); H = default_rng(7).standard_normal((200, 200)). The reference is the exact Ding–Sun–Sun–Toh formula with quotients:
B = G+ ∘ sym(A) + G− ∘ skew(A) + diag(J diag A), with A = UᵀHV.
Result: the jvp's relative error against it is 0.26. With 100 values at 1 + 5e-6·i plus 100 at 2 + 0.01·i (λ = 1 + 2.5e-4) it is 0.082. At spacing 2e-5 (λ = 1 + 2e-3) it is 3e-16. Quotients across 1e-3 would lose only about eps/1e-3 ≈ 2e-13.
Design.
- Keep the chained clusters for the direction d of the JVP (row_jvp). That part needs whole clusters.
- Change only the choice per pair: the averaged limit is used when cid[i] == cid[j] and |s_i − s_j| ≤ band; otherwise the quotient (z_i − z_j)/(s_i − s_j).
- Pairs within the band are always in the same chained cluster, since every adjacent gap between them is ≤ band. The condition is therefore exactly "|s_i − s_j| ≤ band".
- A quotient over a gap above the band loses at most eps·|z|/band ≈ eps^(2/3), as the header already argues.
- G− (den == 0) and r are unchanged.
- Update the header's sentence "Pairs closer than eps^(1/3) max |s| count as tied" to say pairs, not clusters, and that d is built on the chained clusters.
Constraints. - Determinism: unchanged. - Allocation: none. - ABI: unchanged. - Clean room: Ding, Sun, Sun and Toh 2018, already cited. - Spec: no departure.
Tests (tests/python/test_spectral.py, or the existing spectral test file): both setups above give a relative error ≤ 1e-8 against the closed form, and the exact-tie tests (repeated singular values) are unchanged.
As implemented. - The pair rule and the header change are as designed. - The two test setups put λ midway between two values: 1 + 5.125e-4 and 1 + 2.475e-4. With λ on a value, it sits on the breakpoint of the scalar map. - The second setup uses 1.01 + 1e-3·i for the spread values. With values up to 2.99 (2 + 0.01·i), the band grows to 1.8e-5, three spacings, so pairs straddling λ within it take the averaged limit across the breakpoint. That limit is off by O(1) there (1.6e-3 relative). This is the band's design, not chaining.
Acceptance. These tests pass; the existing spectral tests are bit-identical except for chained clusters.
T8. Anisotropic TV: serial structure()¶
- Kind: speed · Severity: low · Target: 1.0.x · Effort: M · Depends on: T1
Problem. structure() (tv_aniso.cpp:889) runs on one thread:
- a union-find over all edges;
- an edges walk that tests (st / s) % L for every entry and follows axis-0 fibers column by column;
- a second walk for the sign sums.
With the B3 setup (D(2048, 11), γ = 0.1, tol 1e-4, threads 0), keep=True adds 74 ms to a 122 ms solve. On D(512, 0) at γ = 0.1, tol 1e-6, it adds about 4 of 30 ms.
Design.
1. Flags. A variant of mark_edges (parallel over fixed fiber chunks, as at :600-650) computes the flat flags with the T1 rule and the forced sides. It needs z, τ, side and stol.
2. Components. Factor out of try_polish (:704-790) the slab union-find, root marking, labelling in root index order and crossing lists into components(g, c, flag, parent, label, cnt) → nc, and use it in both places. Its roots are the smallest index of each component, so the labels equal the serial ones today.
3. Counts and sign sums.
- Counts |C| (inv) per slab, with crossing components done serially.
- Sign sums S_{C,a}: each slab walks its own edges in index order and adds contributions for non-crossing C directly. Entries of crossing components are listed (in parent's storage, as try_polish does) and their edges replayed serially in slab order. The same goes for axis-0 edges between slabs.
- The order is fixed by the shape. Without edge weights the terms are ±1 and the sums are exact. With weights the bits may differ from today but not across nt.
4. v.sign is written per edge and has no reduction.
Constraints.
- Determinism: slabs are fixed by the shape, and crossing work is serial in slab order.
- Allocation: none (uses u and the polish scratch, as try_polish does).
- ABI: unchanged.
- Spec: no departure.
Tests.
- The threading test in test_operators.py with keep=True on TVAniso 2D/3D, with and without edge weights. Assert jvp, vjp, grads and kinks are bit-identical across threads ∈ {1, 2, 5, 15}.
- Labels from the new code equal the old serial labels on 20 random 64² cases (a C++ test exposing nc and the labels through branch_image with side = null).
Acceptance. Add B3 with this change (or with T1). uv run python bench/bench_python.py, section "Anisotropic TV, saved structure": the keep=True minus keep=False difference goes from 74 to ≤ 20 ms, and keep=False does not change.
As implemented. As designed, with these changes.
- Edges. edge_codes applies the T1 rule over the fixed fiber chunks of mark_edges. It writes the flat edges as bit a of a flag byte (in the labels' memory, as the polish does) and the sign of every other edge in two bits per axis of a code byte. The chunk function code_chunk takes restrict pointers and has no branches on the data. Through the lambda's captures every byte store reloaded them. With branches, a 16 × 32 × 32 random walk, where half the edges are flat, cost 5–9 ns an edge; it now costs about 0.7.
- Components. components() is try_polish's slab union-find, labelling and crossing marks, factored out and used in both places. Two changes made it faster for the polish as well:
- the root pass takes the root of an entry's parent when the parent is in the same slab, instead of chasing the chain;
- the slab joins run by rows, without an i % n per edge.
Neither changes the components or the labels. - Sums. The sums are gathered rather than scattered. Each entry adds 1 and, along every axis, the term of its forward edge minus that of its backward edge, when the edge joins another component. Each slab's thread sums runs of entries with the same label in registers: - a run of a component that does not cross is added to it directly; - a run of a crossing one becomes a record, and the records are added serially in slab order, followed by the axis-0 edges into each slab.
On the 2048² image 4.17 M of the 4.19 M entries are in crossing components (the background). A serial pass over their entries took 15 ms; the records take 0.15 ms.
- jump_bound. It runs by rows: row_sum for A, and per-row ranges and maxima reduced in row order. The per-axis maxima of the edge weights are computed once, at construction. A's bits may differ from the serial sum's, but they do not depend on the thread count.
- Scratch. The structure uses u and what follows it in the working set: 3N doubles for the parents, runs and codes, then 4 rows() doubles for jump_bound, or the slab counts and k + 1 sums per entry, at most 7.5 N + 4 doubles in all. The working set has 10 N + P + 8 doubles after u, so image_bytes is unchanged. Slab loops take about kChunk entries per piece, so small images run on one thread.
- Labels and bits. The C++ test that exposes the labels was not written: label is not reachable through the C API. Instead, a script compared the derivatives with master's library on 34 cases. They were 20 random 64² images, 512², 256² at tol 1e-3, rectangles, 3D, tiny γ (the certified path), inactive axes and four with edge weights. It hashed jvp in x and in γ, vjp with its gradients, kinks, and jvp and vjp under branches. Every case without edge weights gave master's bits, and every case gave the same bits at 1, 2, 5 and 15 threads. With edge weights the sums are added in another order, at most 2e-16 relative.
- Tests. In test_tv_aniso.py:
- test_structure_threads checks bit identity across 1, 2, 5 and 15 threads on 1024² and 64³, with and without edge weights; every slab is then a thread's piece.
- test_derivative_bits checks the hashes recorded with master's serial structure() on a 64² and an 8 × 12 × 10 image.
Also run: ASan, UBSan and TSan on test_core, a TSan run of 1024² and 64³ on 8 threads, an ASan sweep of 18 tiny and odd shapes with kinks and branches, and 90 000 sanitized fuzz inputs.
- B3. aniso_keep() in bench/bench_python.py, titled "Anisotropic TV, saved structure": D(512, 0) at tol 1e-6 and D(2048, 11) at tol 1e-4, γ = 0.1, threads 0.
- Measured. Apple M5 Pro, 15 threads, two rounds alternating master's build and this branch's, best of five, in ms:
| Case | keep=False before → after | keep=True − keep=False before → after |
|---|---|---|
| D(2048, 11), tol 1e-4 | 77.9–79.9 → 76.9–77.6 | 101.0–102.4 → 8.5–8.6 |
| D(512, 0), tol 1e-6 | 29.9–30.0 → 29.6–29.7 | 5.6–6.3 → 0.6–1.1 |
| 64³ random walk, tol 1e-6 | 13.2–13.4 → 13.2–13.3 (best of 15, three rounds) | 8.2–8.4 → 1.1–1.3 |
| 64³, edge weights | 20.5–21.2 → 20.7–21.3 | 10.2–11.2 → 1.7–1.8 |
| 16 × 32 × 32 | 6.4–6.5 → 6.3–6.4 | 0.70–0.72 → 0.26–0.31 |
| 256², tol 1e-6 | 13.6–13.7 → 12.4 | 1.2–1.3 → 0.09–0.35 |
| 16² | 0.048 → 0.037 | 0.005–0.012 → 0.004 |
On master the difference was 98–102 ms here, not the 74 ms of the audit's measurement.
T9. Graph and tree TV: overflow of the value range¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: T5 (helper)
Problem. In GraphTV::solve, eps = max(tol·(xmax − xmin), 32·eps·scale) (graph_tv.cpp:226). When xmax − xmin overflows, tol = 0 gives 0·∞ = NaN, every part counts as done, and every node is set to its mean.
Reproducer: the path 0–1–2–3 with x = [1e308, −1e308, 5e307, −5e307], γ = 1e306.
- GraphTV returns [0, 0, 0, 0] with converged = True.
- TreeTV returns about [9.9e307, −9.8e307, 4.8e307, −4.9e307], but its gap is NaN.
Design.
- Apply T5's rescale_exponent in GraphTV::prox_img (:177) and TreeTV::prox_img (tree_tv.cpp:254) after read_input, solving on d̂ = 2^-e d with γ̂ = 2^-e γ (exact). Then z and q are multiplied by 2^e, the gap becomes ldexp(ĝap, 2e), and the saved flow (graph TV's fsave) is multiplied by 2^e, so that kink tests compare it with the unscaled γw_e.
- structure() runs on the scaled values; its regions are scale-free.
- If T6 is not in, γ̂w_e can still overflow; T6's handling of infinite capacities covers that.
Constraints. As T5: bit-identical when e = 0, no allocation, no ABI change.
Tests.
- The path above: GraphTV equals TreeTV to 1e-15 relative.
- Both report converged, and neither gap is NaN.
- GraphTV(256, G(16, 16)) at x·2^1000 and γ·2^1000 (x = default_rng(5).standard_normal(256), γ = 0.3) equals 2^1000·(the unscaled result) to 1e-14 relative.
As implemented.
- rescale and unscale in edge_tv.hpp serve both operators.
- As in T5, the rescale is skipped when a positive capacity would underflow at the new scale.
- A1's step range bounds γ by 2⁹⁶⁰/max w, so the reproducer's γ = 1e306 is now FP_EPARAM, and the tests use 1e288 and 2⁹⁰⁰ in place of 1e306 and 2¹⁰⁰⁰. Near DBL_MAX every capacity is below an ulp of x, so z = x to rounding except at entries far smaller than the rest.
- GraphTV's accuracy is 32 ulps of max |x|, so the mixed-magnitude test checks its small entries at that scale; TreeTV's small entries are checked against the exact reference at 1e-14.
- The gap is ĝap·2²ᵉ and may be inf, as in T5.
- Kinks use the unscaled saved flow, and they agree at 2⁹⁰⁰ and 2⁻¹⁰⁰⁰.
Acceptance. The tests pass and the existing GraphTV/TreeTV tests are bit-identical.
T10. Docs: graph TV derivatives with tol > 0¶
- Kind: docs · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. The GraphTV docstring (bindings/python/fastprox/__init__.py:867) and docs/api.md say that "jvp and vjp are exact". With tol > 0, parts whose interval falls below tol·(max x − min x) are frozen at their mean and become regions of the saved structure. The derivative is then that of the frozen structure.
Setup: GraphTV(2304, G(48, 48)); x = kron(default_rng(12).random((6, 6)), ones((8, 8))) + 0.1·N(0,1) (same generator), flattened; γ = 0.2. The jvp at tol = 1e-2 differs from the jvp at tol = 0 by 0.31 relative. At tol ≤ 1e-4 the two were identical on this input.
Design. Replace the sentence in the docstring, in docs/api.md and in the fp_op_graph_tv comment of fastprox.h with:
"With tol = 0 (the default) jvp and vjp are exact: averages over the regions where z is constant. With tol > 0 the solver may stop a part at its mean before it is split, and the derivatives are those of the regions it stopped at, accurate only as far as those regions are the solution's; use tol = 0 when derivatives matter."
Constraints. Docs only.
Tests. None beyond the docs build (mkdocs build --strict).
As implemented. As designed, in the GraphTV docstring, the fp_op_graph_tv comment and the derivatives bullet of docs/api.md.
Acceptance. The three texts are updated, and the CHANGELOG has an entry under [Unreleased].
P. Python layer¶
Audit of the Python layer (bindings/python/src/module.cpp, bindings/python/fastprox/*.py) on
2026-10-22, on Apple M5 Pro with Python 3.13, numpy 2.5.4, torch 2.14.1 and jax 0.11.2. Unless
stated otherwise, inputs are float64 from np.random.default_rng(0).standard_normal, and times are
the best of 5–7 repeats, measured with timeit. torch/JAX tests run with uv run --group interop.
P1. Second-order reverse mode in the PyTorch wrappers (_Prox, _DeviceTV, _SpectralProx)¶
- Kind: correctness · Severity: high · Target: 1.0.0 (raise), 1.0.x (full) · Effort: S + L · Depends on: none
Scope (decided 2026-10-22). For 1.0.0: remove @once_differentiable from
the three backwards, and when torch.is_grad_enabled() (a backward under
create_graph=True) raise RuntimeError("fastprox: second-order
differentiation through the prox is not supported yet (planned for 1.0.x);
use create_graph=False"). Tested by torch.autograd.grad(..., create_graph=True)
and torch.autograd.functional.hvp raising, for _Prox, _SpectralProx and
_DeviceTV, while first-order backward is unchanged. The full design below
lands in 1.0.x.
Problem. The three torch.autograd.Functions in torch.py (_Prox.backward, :117;
_SpectralProx.backward, :207; _DeviceTV.backward, :411) rely on @once_differentiable to
refuse double backward. In torch 2.14 that decorator does not raise. It hangs a detached Error node
on the gradient, so any second-order term through the prox is silently zero. With
torch.manual_seed(0); x = torch.randn(6, dtype=torch.float64) and
L(x) = (fpt.prox(fp.Soft(0.4), x, 0.8)**2).sum() + (x**3).sum(), the Hessian diagonal (row by row
from torch.autograd.grad with create_graph=True) is
[9.246 -1.7606 -13.0727 3.4106 -6.5071 -8.3916]. The torch-native reference is
[11.246 -1.7606 -11.0727 5.4106 -4.5071 -6.3916], and torch.autograd.functional.hvp is wrong the
same way. With the prox alone the user gets a misleading "differentiated Tensor ... not used in the
graph" error. fpt.tv_iso also returns from a double grad with no error.
Design.
Which second derivatives exist. The backward (gx, gγ, gθ) = (J_xᵀg, ⟨g, ∂z/∂γ⟩, J_θᵀg) is
linear in g. Its derivative with respect to g is the JVP, which is exact for every operator. Its
derivatives with respect to x, γ and θ are second derivatives of the prox. For a large class of
operators these vanish almost everywhere, because J_x is locally constant (the prox is piecewise
affine in x). Each operator class gets a class attribute
_jx_constant: frozenset[str] # variables in which J_x is locally constant a.e.; "x", "gamma", parameter names
_ignores_gamma: bool = False # projections: z does not depend on gamma
Both are filled from the measured table below. For each CASES entry of
tests/python/test_operators.py (rows of length 10, batch 6, x scaled by 1.5), and for TVAniso and
TVIso on (3, 8, 9) at tol 1e-12 and GraphTV and TreeTV on a 12-node chain, op.jvp(ctx, d) in a
fixed direction d was compared after a relative perturbation of 1e-6 of x, of gamma, and of
θ. "Constant" means a relative change ≤ 1e-9. Otherwise the change was 0.02–4 × 1e-6, first order.
ALL means {"x", "gamma"} ∪ every name in _layout and every derived field (lam of Soft, L1Box
and Lq).
| Class | _jx_constant |
_ignores_gamma |
|---|---|---|
| Soft, L1Box, LInf, Max, SumLargest, TV1D, L0, TVAniso (with and without edge weights), GraphTV, TreeTV | ALL | no |
| Box, NonNeg, Simplex, L1Ball (any weights; its only parameter is the radius), Isotonic without weights | ALL | yes |
| TV1DBall | {"x","gamma","alpha"}, plus "weights" when the weights are a scalar |
yes |
| Hyperplane, HalfSpace | {"x","gamma","b","lower","upper"}, plus "a" when a is None or a scalar |
yes |
| Isotonic with weights | {"x","gamma"} |
yes |
| ElasticNet, Huber, SCAD, MCP (weighted or not) | {"x"} |
no |
| L2Ball, SOC | {"gamma"} |
yes |
| GroupL2, TreeL2 (and sparse_group), GroupHuber, GroupSCAD, GroupMCP, LogBarrier, Lq, LogPenalty, TV1DL2, TVIso, Spectral | {} |
no |
Rule for each block. The block (u, v) is the derivative of the backward output for u with
respect to the input v.
(·, g): exact. It is computed by the JVP.(x, v)and(v, x), forv ∈ {x, gamma, θ_k}: zero whenv ∈ _jx_constant, and also whenv = gammaand_ignores_gamma. Otherwise it is unavailable.- Blocks among
gammaandθ(the pairs(γ,γ),(γ,θ_k)and(θ_j,θ_k)): zero when everyγin the pair comes with_ignores_gamma. Otherwise they are unavailable in 1.0.0. A later refinement is a "jointly affine" flag, under which(θ_j, θ_k)vanishes for the Simplex radius, the Box bounds and similar. - An unavailable block yields NaN, never an error and never a silent zero. This is the convention the
JAX wrapper already uses for missing parameter gradients (decisions table, 2026-10-18). The NaN must
depend on the incoming cotangent:
nan_block = w.sum() * torch.full(shape, nan). If it were a constant, it would vanish undertorch.autograd.functional.hvp's double-backward trick and the result would be silently wrong. A prototype with a constant NaN returned GroupL2's HVP 0.67 off and finite. - Rejected: raising in
backwardwhen an unavailable block's input needs grad.ctx.needs_input_gradis fixed at forward time, so the error would also fire for an HVP that only differentiates with respect to an upstream parameterp. That case is exact even for curved operators (see the prototype result below). - Rejected:
torch._C._will_engine_execute_node, which is private.
Functions. Two mutually recursive Functions sit over a state object. _VJPFn computes
g ↦ (gx, gγ, gθ) and _JVPFn computes (dx, dγ, dθ) ↦ dz. Each one's backward applies the
other, so every order of reverse differentiation through the linear part works. A single _VJPFn
whose backward returns a plain JVP tensor is not enough: functional.hvp differentiates the
backward of the backward, which is third order.
Prototype result. A minimal version of this design was checked before writing this spec. It
handled scalar steps and no parameters, with _VJPFn and _JVPFn over op.vjp and op.jvp, and
NaN blocks proportional to the cotangent.
- Setup:
torch.manual_seed(0),xandv = torch.randn(6), andL(y) = (prox(y)**2 * torch.arange(1., 7.)).sum() + (y**3).sum(). - Reference: central differences of the first-order gradient,
(∇L(x+εv) − ∇L(x−εv))/2εwithε = 1e-6. - Soft(0.4), TV1D(0.6) and Simplex(1.0) at γ = 0.8: both
grad(⟨∇L, v⟩, x)andtorch.autograd.functional.hvpmatched the reference to 1.4e-9 to 1.9e-9 (max abs). - GroupL2(0.5), a curved operator: both returned NaN.
-
With L2Ball(0.7), the parameter-only second derivative
∂⟨∇_x (prox(x)·p).sum(), v⟩/∂pequalledop.jvp(ctx, v)exactly (difference 0.0).class _State: # no tensors inside (avoids reference cycles; see Constraints) op, fctx, spec # fctx: the fastprox Context (or SpectralContext / _tvt.Structure) x_meta = (shape, dtype, device); step_meta = None | (shape, dtype, device); value_meta = [...] need = (need_x, need_step, need_values...) # the first Function's needs_input_grad[1:] def vjp(self, g) -> (gx, gstep | None, [gparam_k | None]) # numpy in, torch out (today's backward body) def jvp(self, dx, dstep, dvals) -> dz # today's _Prox.jvp body; dvals via spec.dparams def curvature(self, cot_by_output, anchor_needs) -> per-anchor NaN-or-None (rule above)
class _VJPFn(Function): forward(ctx, st, g, x, step, *values): # x, step, values are anchors only, never read ctx.set_materialize_grads(False); ctx.st = st; ctx.save_for_backward(x, step, *values) return st.vjp(g) # tuple; None entries for outputs not needed backward(ctx, ggx, ggstep, *ggp): anchors = ctx.saved_tensors gg = _JVPFn.apply(st, ggx, ggstep, *ggp, *anchors) if ctx.needs_input_grad[2] else None return (None, gg, *st.curvature((ggx, ggstep, *ggp), ctx.needs_input_grad[3:])) jvp(ctx, _st, dg, dx, dstep, *dvals): # forward-over-reverse return st.vjp(dg) + NaN/zero curvature for anchor tangents (same rule)
class _JVPFn(Function): forward(ctx, st, dx, dstep, *dvals_and_anchors): ... save anchors; return st.jvp(dx, dstep, dvals) backward(ctx, w): # adjoint: (dx, dstep, dvals) <- _VJPFn outs = _VJPFn.apply(st, w, anchors) if any(ctx.needs_input_grad[1:1+1+P]) else (None,)(2+P) return (None, *outs, *st.curvature((w,), ctx.needs_input_grad[anchors]))
The adjoint pairs are fixed by the first-order code:
x:op.vjpagainstop.jvp(dx).γ(scalar):grads.gammaagainstdgamma = float(dstep.sum()). The sum is needed becausestep_gradbroadcasts one value to the step's shape whenprod(shape) == 1.γ(per row):step_gradagainstspec.step_tangent.θ:spec.param_grads(reshape, or sum when the value has size 1) againstspec.dparams(_flatten, which broadcasts).- Derived fields: a new
Operator._derived_dparams(name, t)returns the flat θ tangent. For Soft, L1Box and Lq,lamenters only as the productλw, so a tangentdλis the weights tangentw·dλ/λ, the adjoint of the existinglam = (w·g_w)/λ.Spec.dparamsandOperator._flattenaccept the derived names through it, which also enables forward mode forlam; today it raises.
_Prox.
forwardstoresctx.st = _State(...)in place of today's six attributes, and callsctx.save_for_backward(x, step if tensor else None, *values).-
backward, with@once_differentiableremoved:def backward(ctx, g): if not torch.is_grad_enabled(): # create_graph=False: today's code path, unchanged return (None, *ctx.st.vjp_first_order(g, ctx.needs_input_grad)) anchors = ctx.saved_tensors # version check happens here only return (None, *_VJPFn.apply(ctx.st, g, *anchors))
Unpacking saved_tensors only under create_graph keeps today's behaviour, in which the input can
be mutated in place after the forward: y = x * 1.0; z = fpt.prox(op, y, 0.8); y.mul_(100) under
no_grad, then z.pow(2).sum().backward() gives today the gradient at the original point. Under
create_graph the second-order graph must attach to the input's history at forward time, so
torch's in-place error is the correct outcome there.
- _Prox.jvp (first-order forward mode) keeps its body. It should return
_JVPFn.apply(st, dx, dstep, *dvals, *anchors) so that its result carries a graph for
reverse-over-forward. Forward-over-forward (nested dual levels) also goes through _JVPFn.jvp.
The implementer verifies that torch accepts a Function.apply inside jvp. If it does not, both
stay unsupported and documented.
- torch.func transforms stay unsupported. They need setup_context, and NumPy calls cannot be
vmapped.
_DeviceTV. Same pattern, with _DeviceTVState wrapping _tvt.Structure.
vjp(c):S.vjp(c, deriv_tol, deriv_max_iter)returns(r, ggamma (B,), gw: Python floats, S.gsw). Convertgwto a tensor ofwt.shape, summed whenwt.numel() == 1, as today.jvp(dx, dgamma (B,), dw, dsw):S.jvp(dx, dgamma, dw as floats (repeated k times when wt has one entry), tol, max_iter, dsw), run undertorch.no_grad()as today._jx_constantis ALL for anisotropic TV (measured as above: constant in x, gamma, weights and edge weights) and{}for isotropic TV.- Anchors are
(x, gb, wt, sw), saved withsave_for_backward. cfg.derivis updated by every solve, as now.
_SpectralProx.
- State
vjp(g)is_derivative(ctx, g, 0, None)plusop._grads. - State
jvpis_derivative(h, dgamma, dparams)with the per-row term from the existingjvp. _jx_constant = {}: U and V depend on X, so only thegpath is exact and thexblocks are NaN.
Contract text. Replace the torch.py module docstring's "not double backward" with:
Second-order reverse mode (
create_graph=True,torch.autograd.functional.hvp/hessian) is exact through the linear part of the backward. Through the prox's own curvature it is exact for operators whose Jacobian is locally constant (the piecewise-affine maps: soft thresholding, the projections onto polyhedra, 1D, anisotropic, graph and tree TV, ...), and NaN where that curvature is not available (smooth maps such as the l2 ball, group norms, isotropic TV and spectral operators, and second derivatives in gamma and the parameters). Each operator lists its_jx_constant.
Add the same paragraph to the PyTorch section of docs/api.md. JAX needs no change: jax.hessian
already raises "Pure callbacks do not support JVP".
Constraints.
- Determinism: unchanged. Every number still comes from
fp_vjpandfp_jvp. - Allocation: none in C.
save_for_backwardkeepsxalive until the graph is freed, as most upstream ops already do. _Statemust not hold tensors. Anchors go throughsave_for_backwardonly. Holding them in the state object creates graph-to-state cycles.- A graph alive at interpreter exit already prints nanobind's "leaked 2 instances" today: a script
that ends holding
torch.autograd.grad(..., create_graph=True)'s result does this. Afterdelandgc.collect()nothing leaks, so it is a shutdown-order artifact and is out of scope here. - ABI: none.
- Departure from docs/spec.md: yes. Decisions-table row:
| 2026-10-22 | The PyTorch wrappers support second-order reverse mode: the backward is a Function whose backward is the JVP (and vice versa), exact through the linear part and through the prox's curvature where its Jacobian is locally constant (per-class_jx_constant); unavailable blocks are NaN, as JAX's missing parameter gradients, rather than an error, sinceneeds_input_gradcannot tell which second derivatives are requested | fastprox/torch.py, fastprox/_interop.py, fastprox/__init__.py, docs/api.md |
Tests. New file tests/python/test_torch_second_order.py, all in float64:
- For every
CASESname whose class has"x"in_jx_constant, and for TVAniso and GraphTV, withL(x) = (prox(x)**2 * arange).sum() + (x**3).sum()and randomx,v: grad(grad(L, x, create_graph)·v, x)andfunctional.hvpboth match central differences of the first-order gradient (ε = 1e-6, abs tolerance1e-6·(1+|fd|)).- For Soft with the Problem's input (
torch.manual_seed(0), 6 entries), the Hessian diagonal equals the torch-native soft-threshold reference (assert_allclose, rtol 1e-12). - For a curved class (GroupL2, L2Ball, TVIso through
fpt.tv_iso, NuclearNorm): the HVP inxis all-NaN. - For the same curved classes, the parameter-only HVP
grad(⟨grad_x (prox(x)·p).sum(), v⟩, p)equalsop.jvp(ctx, v)exactly. - Mixed blocks: ElasticNet with a tensor gamma requiring grad gives NaN for
∂(gx)/∂γ. Soft gives finite values matching finite differences for∂(gx)/∂γ(zero) and∂(gx)/∂w. - Forward-over-reverse: a forward-AD dual over
grad(L, x, create_graph=True)matches the HVP. - The in-place mutation case from the
_Proxparagraph still passes undercreate_graph=Falseand raises torch's in-place error undercreate_graph=True. - A flag-consistency test parametrised over
CASES(the perturbation measurement above) asserts that each class's_jx_constantmatches the measured local constancy. Adding an operator without the attribute fails, which extends the CLAUDE.md "Adding an operator" checklist.
Acceptance. uv run --group interop pytest -q tests/python/test_torch_second_order.py passes,
and so does the existing torch suite. In uv run --group interop python bench/bench_python.py, the
line fpt.prox(Soft) fwd+bwd of the framework section (defined in P11) stays within 3% of its
value before the change (22.9 µs on Apple M5). The other sections show no regression.
P2. Python side of overlapping out/dx/g buffers¶
- Kind: correctness · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: C1 (the C overlap rule)
Problem. _output (__init__.py:219) passes any out through, and the out= of jvp
(:357) and vjp (:368) is not checked at all. The C core detects only x.data == out.data, so
overlapping views give silently wrong results. Max error against the correct result, for x of
shape (50, 50), tv1d at γ = 0.5, soft at λ = 0.4, simplex at radius 1:
| Call | out=x.T (square x; same data pointer, other strides) |
out=x[::-1] |
|---|---|---|
| tv1d | 2.81 | 2.63 |
| soft | 3.69 | 4.14 |
| simplex | 1.0 | 1.0 |
With TV1D(0.6) at γ = 0.8 on (6, 50), vjp(c, g, out=g[::-1]) is off by 4.38 and
jvp(c, dx, out=dx[:, ::-1]) by 4.06. Exact
out=x is correct.
Design. One helper in __init__.py, used everywhere a C call takes an input buffer and an output
buffer:
def _stage(src, dst):
"""src, or a copy of it when dst overlaps it without being the same array."""
if dst is None or dst is src or not np.may_share_memory(src, dst):
return src
if (src.__array_interface__["data"][0] == dst.__array_interface__["data"][0]
and src.shape == dst.shape and src.strides == dst.strides):
return src # the same layout through another view object: the C alias case
return src.copy() # C order, the same dtype
- Call sites:
Operator._run,_Pointwise._run_tie,_ImageOperator._iterand_once, after_output:x = _stage(x, z).- The
stepsarray from_stepsgetssteps = _stage(steps, z), for a per-row gamma that is a view ofout. Operator.jvp:dx = _stage(dx, dout).Operator.vjp:g = _stage(g, gx).TVIso.jvpandvjpinherit these.np.may_share_memoryis a bounds check costing 76 ns, the only cost on the common path. It conservatively copies interleaved views that do not overlap, such asx[::2]without = x[1::2], which is correct, only a copy.- Rejected:
np.shares_memory. It is exact but can be exponential. - Python also validates
outbefore any C call, so the CFP_EINVALis unreachable from Python: outmust be an ndarray, writeable, of the input's shape and dtype (also forjvp/vjp'sout, which today get no check). RaiseValueError("out must be a writeable ndarray with the shape and dtype of the input").outmust have no zero stride on an axis of extent > 1. RaiseValueError("out must not have zero strides").- Docstrings of
prox,prox_conj,jvpandvjpgain: "outmay be the input itself or overlap it; an overlapping input is copied first."
Constraints.
- Determinism: unchanged. A staged copy is bit-identical input.
- Allocation: one input-sized copy, only on overlap. The C no-allocation rule is unaffected, since the copy happens in Python before the call.
- ABI: none.
- Departure: none on the Python side. The C decision row is C1's.
Tests. New tests/python/test_out_overlap.py, parametrised over NAMES from test_operators.py
plus TVAniso, TVIso and GraphTV:
- Views on a square
(10, 10)batch:out=x.T,out=x[::-1],out=x[:, ::-1], shifted overlap (buf[1:]againstbuf[:-1]), interleavedx[::2]againstx[1::2], andout=x.view(). assert_array_equal(bitwise) with the result into a fresh buffer.- The same for
jvp(out=dx[::-1])andvjp(out=g[::-1]). - Read-only
out, wrong-dtypeout, and a writeable stride-0outmade withas_stridedraiseValueErrorwith the messages above.
Acceptance. The test passes. In uv run python bench/bench_python.py, the overhead line
Soft().prox(x, out=out) (added in P11) stays ≤ 1.10 µs (1.05 µs today on Apple M5), and the existing
line TV1D().prox(x, out=out) grows by ≤ 0.1 µs.
P3. Serialise calls on one Context¶
- Kind: safety · Severity: medium · Target: 1.0.0 · Effort: S · Depends on: none
Problem. The GIL is released in prox, jvp and vjp, but a Context (__init__.py:110)
shares one fp_workspace with no guard. Its per-thread scratch, its branch override (_branch,
:166) and the derivative tolerance (TVIso.jvp/vjp, :985, set in a separate call) are all
shared state. With 4 threads × 30 calls on one context, made with keep=True, threads=1 and
compared bitwise with serial calls (each thread passes its own random g of the input's shape):
- Inputs: TVIso(2, 0.3) on
(4, 64, 64)at tol 1e-8, and Soft(0.4) on(256, 4000)withkink_tol=0.5, all at γ = 0.8. -
Results:
-
TVIso vjp: 80/80 results wrong, worst relative error 3.9e262. Serial repeats are bit-identical.
- Soft with
branch=±1: 118/120 wrong.
jvp and vjp look read-only, so nothing warns the user.
Design.
- Add
"lock"toContext.__slots__, withself.lock = threading.Lock()in__init__(18 ns). - Hold
with ctx.lock:around each of these sequences: - In
Operator._runand_run_tiewithkeep=True: the_C.proxcall and thectx.info,ctx.conj,ctx.stepwrites. - In
_ImageOperator._iter:set_warm,set_dual,_C.prox,get_dualand the info writes. - In
Operator.jvpandvjp: the whole_branchcontext and the_Ccall. - In
TVIso.jvpandvjp:set_deriv_tol, the call, andderiv_info, as one critical section. To avoid re-entry, call a private_jvp_locked/_vjp_lockedthat assumes the lock is held. Context.kinks.Spectral._coefficientsand_grads:inner.lockaround_C.spectral_jvpand the inner vjp. The firstfp_spectral_jvpcall also sizes scratch in the workspace.threading.Lock.acquirereleases the GIL while it waits, and the C calls never take the GIL, so this cannot deadlock.- Contract text in the
Contextdocstring: "Calls on one context are serialised; for parallel derivatives use one context per thread." - Rejected: raising when the context is busy, which turns a correct program into an error depending on timing.
- Rejected: one workspace per call, which allocates per call and breaks the no-allocation design.
Constraints.
- Determinism: results become bit-identical to serial calls.
- Allocation: one lock per Context.
- ABI: none.
- Departure: none. The spec says one workspace per concurrent caller, and the lock enforces it.
Tests. Add test_context_concurrent to tests/python/test_threads.py (or a new file), with the
Problem's setup:
- Workloads: TVIso
(4, 64, 64)at tol 1e-8, interleaved vjp and jvp; Soft(256, 4000)withbranch=+1and-1alternating; Spectral NuclearNorm vjp. - 4 threads × 30 calls each.
- Every result
array_equalto the serial reference.
As implemented.
- A re-entrant lock (threading.RLock) instead of a plain lock with _locked helpers. TVIso.jvp calls the base jvp inside its own critical section, and Spectral holds the inner context's lock across its core call and the inner vjp. The two locks differ by about 10 ns.
- The lock covers:
- _run, _run_tie and _iter with a context;
- jvp and vjp, the TVIso derivative sections, Context.kinks;
- Spectral jvp and vjp.
- The cost. The _branch context manager used to run even when no branch was given. It is now skipped then (a nullcontext), which pays for the lock: Soft.jvp on 8 entries went from 0.64 to 0.65 µs. With the lock alone it was 0.73 µs, at the edge of the budget.
- The test passed 20 runs in a row.
Acceptance. The test passes 20 runs in a row. In uv run python bench/bench_python.py, the
overhead line Soft jvp(ctx, x) (added in P11) grows by ≤ 0.1 µs (0.78 µs today on Apple M5).
P4. Release the GIL in operator constructors and other data-sized C calls¶
- Kind: scaling · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. make (module.cpp:105) calls the fp_op_* constructors with the GIL held.
GraphTV(2_000_000, edges) took 255 ms, where the edges are 8e6 random pairs from
rng.integers(0, n, (4n, 2)) with self-loops dropped. During that time a thread that appends
time.perf_counter() and sleeps 1 ms made no progress: the max gap was 255.2 ms. docs/spec.md says the GIL is "Released ... for every call into the C
API". workspace (:152), get_dual, set_dual, kinks and set_branch also hold it, though
each measured ≤ 0.1 ms.
Design.
make:fp_status s; { nb::gil_scoped_release r; s = f(&op->p); } check(s);. The lambdas capture onlyfp_arrayviews and raw pointers intonb::ndarrayarguments, which the caller's frame keeps alive. Exceptions are thrown bycheckafter the GIL is reacquired.- The same pattern goes in
workspace,get_dual,set_dual,kinksandset_branch. - Trivial getters keep the GIL:
nparams,name,spectral_check,abi_version,version,default_threads,set_warm,set_deriv_tol,deriv_info. A release and reacquire costs about as much as the call.
Constraints.
- Determinism and allocation: unaffected. The constructors allocate, as now.
fp_opconstruction has no global state (fastprox.h: no global error state).- Departure: wording only. Docs/spec.md, Python bindings, GIL: "Released for every call that does
work proportional to its data; constant-time getters keep it." Row:
| 2026-10-22 | The Python bindings release the GIL in every C call whose cost grows with its data (prox, derivatives, value, operator constructors, workspace creation, dual and kink copies); constant-time getters keep it, since releasing costs as much as the call | bindings/python/src/module.cpp, docs/spec.md |
Tests. tests/python/test_threads.py::test_constructor_releases_gil:
- A ticker thread increments a counter every 1 ms while the main thread builds
GraphTV(5e5, 2e6 random edges). - Assert the counter rises by at least
0.5 · construction_ms. - Skip when the construction takes under 20 ms.
As implemented. As designed, and fp_prox_steps's path was already released. GraphTV construction on 5·10⁵ nodes is unchanged (31–33 ms both ways). The test asserts a quarter of a tick per millisecond, allowing for the scheduler's sleep granularity.
Acceptance. The test passes. Construction time is unchanged within 2%.
P5. Keep float32 for non-ndarray array-likes¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: none
Problem. _input (__init__.py:216) converts every non-ndarray with
np.asarray(x, dtype=np.float64). float32 torch tensors, JAX arrays, memoryviews and array('f')
come back as float64 (fp.tv1d(jnp.asarray(x32), 0.5).dtype == float64 for x32 a float32 array of 50
entries), at the cost of
a copy and double the arithmetic.
Design.
def _input(x, copy):
if not isinstance(x, np.ndarray):
x = np.asarray(x) # zero-copy for buffer/DLPack/__array__ providers
if x.dtype in (np.float32, np.float64):
return x
if np.iscomplexobj(x):
raise TypeError("fastprox operates on real float32 and float64 arrays, got complex input")
return x.astype(np.float64) # lists of numbers, ints, bools: as before
... # ndarray branch unchanged
- Python lists and scalars still give float64.
- Complex array-likes now raise instead of discarding the imaginary part with a
ComplexWarning. - Docstring and docs/api.md (Python, inputs): "Non-NumPy inputs are converted with
numpy.asarray; float32 and float64 are kept, other real types become float64." - CHANGELOG under Changed.
ProxOp._shaped(pyproximal.py:72) also converts complex to float64 silently. It should raiseTypeErrorfor complex, and convert float16 to float64 rather than lettingop.proxraise.
Constraints. No effect on determinism or allocation (one copy fewer). No ABI change. No departure: spec's "converted only when the caller sets copy=True" is about ndarrays; record nothing.
Tests. tests/python/test_inputs.py:
- A float32 CPU torch tensor, a JAX float32 array,
memoryview(np.float32 array)andarray.array('f'):fp.tv1d(...)has dtype float32 and isarray_equalto the ndarray result. [1, 2, 3]gives float64.- A complex list raises
TypeError. - Skip the torch and JAX cases when not installed.
As implemented. As designed, and the PyProximal adapter also converts float16. The tests cover memoryview, array('f'), PyTorch, JAX, lists, booleans, complex and the adapter.
Acceptance. The test passes. bench/bench_python.py shows no regression.
P6. Reject unknown names and wrong sizes in dparams¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: S · Depends on: P1 (derived names via
_derived_dparams)
Problem. Operator._flatten (__init__.py:386-393) silently ignores keys of a dparams dict
that are not in _layout. With op = fp.TV1D(0.6) and _, c = op.prox(x, 0.8, keep=True),
op.jvp(c, None, 0.0, {"bogus": 1.0}) returns a zero tangent, so a typo goes unnoticed. A flat dparams of the wrong length reaches C and comes
back as "mismatched shape or parameter length".
Design.
- In
_flatten, checkunknown = set(dparams) - {layout names} - {derived names}. If it is non-empty, raiseValueError(f"{type(self).__name__} has no parameter {sorted(unknown)}; its parameters are {names}"). - Derived names (
lamfor Soft, L1Box, Lq) are mapped through_derived_dparams(P1). - A dict value whose size is neither the field's length nor 1 raises
ValueError(f"dparams[{name!r}] has {v.size} entries, {name} has {length}"), instead of numpy's broadcast error. - A flat array whose size is not
self.nparamsraisesValueError(f"dparams has {n} entries, {type(self).__name__} has nparams={k}"). Spectral._flattendelegates, so it is covered.
Constraints. None in C. No departure.
Tests. In tests/python/test_operators.py:
- An unknown key raises.
- A wrong field size raises.
- A wrong flat length raises.
{"lam": t}on Soft equals{"weights": w·t/λ}to 1e-15.
As implemented.
- _flatten checks names and sizes as designed.
- Full P1 has not landed, so the derived names come from a small hook added here, Operator._derived_dparams(name, value):
- lam maps to weights · t/λ for Soft, L1Box and Lq;
- radius maps to (lower, upper) = (−t, t) for SpectralNormBall.
- Spectral derivatives now go through Spectral._flatten. They called the inner operator's _flatten directly, which skipped the derived names.
Acceptance. The tests pass.
P7. Clear error messages at the Python boundary¶
- Kind: correctness · Severity: low · Target: 1.0.0 (a), (b), (d); 1.1 (c-parity) · Effort: S · Depends on: P2 (
outvalidation)
Problem. These calls produce unhelpful errors. x has shape (6, 50), op = fp.TV1D(0.6)
unless stated otherwise, and torch inputs are torch.randn(4, 20) in the named dtype:
- (a) torch float16, int or complex input gives
TypeError: Cannot interpret 'torch.float16' as a data type(torch.py:84→_interop.check_dtype). - (b) A read-only
out,axis=1.0, oroutgiven as a list produces a nanobind "incompatible function arguments" signature dump.threads=-1is silently taken as the default (Cresolve_threads). - (c) A one-shot function given an array gamma (
fp.tv1d(x, np.full(6, .5))) reports "only 0-dimensional arrays can be converted to Python scalars" (_once,:1326), althoughOperator.proxaccepts one step per row. Withaxis=5, a one-shot reports the generic C "invalid argument", because_oncenever calls_batch_shape. - (d)
branch=2passes_branch's int8 round-trip check and returns C's "invalid operator parameter or step".
Design.
- (a) In
torch._applyand_device_tv, before anything else:if x.dtype not in (torch.float32, torch.float64): raise TypeError(f"fastprox operates on float32 and float64 tensors, got {x.dtype}"). - (b) In
Operator._run,_run_tie,_iterand_once: axis = operator.index(axis)andthreads = operator.index(threads).if threads < 0: raise ValueError("threads must be >= 0 (0: default)").outvalidation as in P2.- (c)
_oncecallsself._batch_shape-equivalent validation and_steps. A non-scalar gamma raisesTypeError("the one-shot functions take a scalar gamma; use the operator class (e.g. TV1D().prox) for one step per row")in 1.0.0. Full parity (one-shot functions accepting per-row gamma) adds API and is a 1.1 option. - (d)
_branch:if not np.isin(side, (-1, 0, 1)).all(): raise ValueError("branch entries must be -1, 0 or 1"). module.cppcheck(s, const char *what)prefixes the C entry point's name, as in"fp_prox: invalid argument", so errors that do reach C say which call failed.fp_strerroris unchanged.
Constraints. None in C. No departure. Docs/spec.md says the messages carry "the text of fp_strerror", and they still do, prefixed.
Tests. In tests/python/test_errors.py (new): each reproducer above raises the named exception
type and match= its message.
As implemented.
- (a), (b), (c) for 1.0.0, and (d), as designed. The one-shot functions also check axis against the input's dimensions.
- check(s, what) in module.cpp names the entry point. Constructors report "operator constructor".
- JAX already raised a clear TypeError for other dtypes.
Acceptance. The tests pass.
P8. Empty batches and NaN or Inf images: one contract across the C, PyTorch and JAX paths¶
- Kind: correctness · Severity: low · Target: 1.0.0 · Effort: M · Depends on: A8 for the NaN certificate; the empty-batch short-circuits stand alone
Problem. The three paths disagree. Setup: γ = 0.5, default weights, NaN and Inf images of shape
(8, 8) (all entries, or a single NaN pixel for the C path).
| Input | C (TVIso/TVAniso/GraphTV.solve) |
fpt.tv_iso/tv_aniso |
fpj.tv_iso |
|---|---|---|---|
empty batch (0, 4, 4) or (2, 0, 4) |
ValueError: invalid argument |
IndexError (_tv_torch.py:214, gamma[0]) |
ZeroDivisionError |
| NaN image | z all NaN, gap 0.0, iters 0, converged=True |
NaN, 1000 iterations, gap NaN, converged=False |
same as torch |
| Inf image | ValueError (documented FP_EINVAL) |
accepted, NaN, 1000 iterations | accepted, NaN, 1000 iterations |
Row operators accept empty batches ((0, 5) and (5, 0) work).
Design. Contract, added to docs/api.md under each iterative operator's "Special values":
An empty batch (any axis of extent 0) returns an empty result with gap 0, 0 iterations, converged true and an empty dual. An image containing a NaN gives a NaN image; its gap is NaN, it takes no iterations and it counts as finished, so it does not by itself make
convergedfalse (A8). The rest of the batch is unaffected. ±Inf is rejected (ValueError), except underjax.jit, where values cannot be checked and an image with an Inf is treated as a NaN image.
- NumPy (
_ImageOperator._iter): ifx.size == 0: - Return
z(the emptyout) andIterInfo(0.0, 0, 0, True, np.empty((ndim, *x.shape))). - With
keep=True, return aContextwhosewsis None. Operator.jvp/vjpandContext.kinkscheck forctx.ws is Noneand return empty arrays andGrads(0.0, zeros(nparams), ...)without calling C.- When the C side accepts empty images, the short-circuit can stay as a fast path.
- NumPy NaN: no Python change beyond the docstring; the certificate (gap NaN, 0 iterations, finished) comes from C (A8).
- torch device (
_device_tv): - If
x.numel() == 0, returnx.clone() + 0 * (gb.sum() + (wt.sum() if wt is not None else 0) + (sw.sum() if sw is not None else 0)). This keeps every differentiable input in the graph, so empty gradients come back as zeros rather than None. - Return the info as above.
if bool(torch.isinf(x).any()): raise ValueError("x contains ±Inf; image operators reject infinite values"). It is one host sync per call; the solve already syncs everyCHECKiterations.- In
_tvt.solve,bad = torch.isnan(xb).flatten(1).any(1). Start withlive = ~bad, setz[bad] = nanandgap[bad] = nan, and skip the loop when nothing is live. Structure(the backward) leaves NaN images' gradients NaN and the others finite.- JAX device (
_device_tv,_tvj.tv): x.size == 0is a static Python check, so it returns as above.- NaN or Inf images get
live = ~badthe same way, inside the traced code withjnp.where.
Constraints.
- Determinism: unchanged.
- Allocation: none in C.
- Departure: docs/api.md "Special values" for the device paths (Inf under
jit). Row:| 2026-10-22 | Iterative operators on an empty batch return an empty result with gap 0 and converged true; a NaN image reports gap NaN and 0 iterations and counts as finished on every path (A8); under jax.jit an image with ±Inf is treated as a NaN image, since values cannot be checked there (the CPU and PyTorch paths reject it) | fastprox/__init__.py, fastprox/_tv_torch.py, fastprox/_tv_jax.py, docs/api.md |
Tests. tests/python/test_special_values.py (new):
- Empty
(0, 8, 8)and(2, 0, 8)for TVIso, TVAniso and GraphTV (numpy),fpt.tv_iso/tv_anisoandfpj.tv_iso(eager and jit): the shapes, the info fields, and a backward that returns empty gradients. - A NaN pixel in image 1 of a batch of 3:
- Image 1 is all NaN,
info.gapis NaN,info.convergedis True anditersequals that of the clean images alone. - Images 0 and 2 are
array_equalto their solo solves (numpy) or within the tolerance (device). - Inf:
ValueErroron numpy and torch; NaN output underjax.jit.
As implemented.
- NumPy. No Python short-circuit was needed: since A3 the C library accepts empty batches. The empty call now also marks its (empty) dual valid, so fp_workspace_get_dual succeeds, and every derivative works on the empty point.
- PyTorch.
- _device_tv returns empty batches before the solve, in the graph of every differentiable input, and rejects ±Inf, at the cost of one host sync.
- _tv_torch.solve zeroes the non-finite images, starts them not live, and sets their z, u and gap to NaN at the end. converged is taken over the other images. The backward leaves the clean images' gradients finite.
- JAX. The same masking is done with jnp.where inside _tv_jax.solve, so it holds under jit. Eager calls reject ±Inf; traced ones cannot, and treat it as a NaN image. Empty batches return early, since shapes are static.
- The JAX test runs at tol 1e-6, since JAX defaults to float32. Its reference is the same 3-image program with the NaN image replaced by a copy of image 0, because a batch of another shape compiles to other float32 reductions and other iteration counts.
- A8's device part is done here.
Acceptance. The tests pass. fpt.tv_iso of a NaN image takes 0 iterations (today 1000).
P9. JAX: compile once per operator instead of dispatching eagerly¶
- Kind: speed · Severity: high · Target: 1.0.x · Effort: M · Depends on: none
Problem. Called eagerly (no outer jit, which is common in notebooks and loops), the JAX
wrappers trace their custom_vjp and pure_callback on every call. Setup: jax_enable_x64, x of
100 float64 entries (32×32 for tv_iso), γ = 0.8 (0.3 for tv_iso), and .block_until_ready() on
every result:
| Call | Eager | Jitted |
|---|---|---|
fpj.prox(fp.Soft(0.4), x[100], 0.8) |
2495 µs | 31 µs |
jax.grad of it |
6233 µs | 48 µs (404 µs when the grad itself is not jitted but the function is) |
fpj.tv_iso 32×32 |
69.8 ms | 0.51 ms |
The per-call _function(spec) closure (jax.py:77) is not the cause: reusing one function costs
2470 µs too.
Design. Cache a jax.jit of the built function, keyed on everything the closure depends on.
-
In
fastprox/jax.py:_CLASS_CACHE: "OrderedDict[tuple, Callable]" = OrderedDict() # LRU, 128 entries _CACHE_ATTR = "_fpj_jit"
def _jitted(spec, build): key = (spec.spectral, spec.conj, spec.threads, tuple(sorted(spec.call.items())), spec.names) if isinstance(spec.op, (Operator, Spectral)): # an instance: cache on it cache = spec.op.dict.setdefault(_CACHE_ATTR, {}) else: # a class with static parameters frozen = _freeze(spec.static) if frozen is None: # unhashable static value: no cache return build(spec) key = (spec.op, frozen) + key cache = _CLASS_CACHE f = cache.get(key) if f is None: f = jax.jit(build(spec)); cache[key] = f if cache is _CLASS_CACHE and len(cache) > 128: cache.popitem(last=False) elif cache is _CLASS_CACHE: cache.move_to_end(key) return f
-
_freeze(static)returnstuple(sorted((k, _fz(v)) for k, v in static.items())), where_fzmaps: - an ndarray to
("nd", v.dtype.str, v.shape, v.tobytes()); - a list or tuple to a tuple of
_fz; - None, bool, int, float, str to itself;
- anything else to None, which disables the cache.
Freezing costs O(bytes), well below building the operator in each callback (spec.build).
- Instance cache lifetime equals the operator's. Operators are immutable after construction, so the
key never goes stale.
- _apply calls _jitted(spec, _function)(x, step, *values). Shape-dependent closure state in
_spectral (m, n, rect and the two ValueError checks) moves into the traced body, which
reads x.shape statically, and the checks stay in _apply before the cache. JAX's jit cache then
handles new shapes and dtypes by retracing.
- Device TV: a module-level _tv_jit = jax.jit(_tvj.tv, static_argnums=(5, 6, 7, 8, 9)), plus a
jitted dual_out. sw=None is an empty pytree, so it is a valid dynamic argument. _device_tv
calls _tv_jit.
- Nested in a user's jit, the inner jit is inlined, so nothing changes there. vmap and grad
compose as before. Under free-threading (P12), a race on the cache dict can at worst compile the
same function twice.
- Rejected: functools.lru_cache on Spec, because static holds unhashable arrays.
- Rejected: caching keyed on id(op), because ids are reused after the operator is freed.
Constraints. No effect on determinism (same callbacks). No C allocation. No ABI change. No departure.
Tests. tests/python/test_jax.py:
- Calling
fpj.prox(op, x, g)twice with the same instance leaves one entry inop.__dict__["_fpj_jit"]. - Two class calls with equal static arrays share a
_CLASS_CACHEentry; different arrays do not. - Results are bitwise equal to the uncached function.
- Spectral and the device TV on two different shapes give correct results.
Benchmark. Add a section jax_wrappers() to bench/bench_python.py, called last from
__main__:
- It starts with
try: import jax, jax.numpy as jnp; import fastprox.jax as fpjandexcept ImportError: print("JAX wrappers: jax not installed, skipped"); return. - It sets
jax.config.update("jax_enable_x64", True). - Each line times one call with the module's
best,.block_until_ready()included, onx = jnp.asarray(np.random.default_rng(0).normal(size=100)):
| Line | Call |
|---|---|
fpj.prox(Soft) eager |
fpj.prox(fp.Soft(0.4), x, 0.8), with the operator built once outside the timed call |
jax.grad(fpj.prox(Soft)) eager |
jax.grad(lambda x: fpj.prox(soft, x, 0.8).sum())(x) |
fpj.prox(Soft) jit |
the same, through jax.jit(lambda x: fpj.prox(soft, x, 0.8)) compiled before timing |
jit(grad(fpj.prox(Soft))) |
the grad, jitted |
fpj.tv_iso 32x32 eager |
fpj.tv_iso(x2, 0.3), x2 of shape (32, 32) from the same generator |
fpj.prox(Simplex) jit 64x1e4 |
jax.jit(lambda x: fpj.prox(fp.Simplex(1.0), x, 1.0, threads=1)) on (64, 10000) |
jit(grad(fpj.prox(Simplex))) 64x1e4 |
jax.jit(jax.grad(lambda x: (fpj.prox(simplex, x, 1.0, threads=1) * x).sum())) |
The module docstring gains: "uv run --group interop python bench/bench_python.py adds the PyTorch
and JAX wrapper sections."
Acceptance. Run uv run --group interop python bench/bench_python.py. The targets are for
Apple M5:
| Line | Before | Target |
|---|---|---|
fpj.prox(Soft) eager |
2495 µs | ≤ 60 µs |
jax.grad(fpj.prox(Soft)) eager |
6233 µs | ≤ 600 µs |
fpj.tv_iso 32x32 eager |
69.8 ms | ≤ 1.5 ms |
fpj.prox(Soft) jit |
31 µs | within 5% |
jit(grad(fpj.prox(Soft))) |
48 µs | within 5% |
The other sections do not regress.
As implemented. As designed, with these differences:
- The key also holds the derivative keywords (deriv_tol, deriv_max_iter), inner_accuracy's captured state, and whether the step can carry a gradient (P11). The callbacks read all three from the closure.
- An unhashable constant makes _fz raise, and _freeze returns None, which disables the cache.
- The device TV. _tvj.dual_in and _tvj.dual_out are jitted as well as _tvj.tv.
- Instance cache lifetime. A cached function refers back to its operator through its spec, a reference cycle that the garbage collector frees with the operator.
- Measured (uv run --group interop python bench/bench_python.py, Apple M5 Pro, µs):
| Line | Before | After | Target |
|---|---|---|---|
fpj.prox(Soft) eager |
1880 | 55 | ≤ 60 ✓ |
jax.grad(fpj.prox(Soft)) eager |
4387 | 497 | ≤ 600 ✓ |
fpj.tv_iso 32x32 eager |
68363 | 712–740 | ≤ 1500 ✓ |
fpj.prox(Soft) jit |
30.9 | 28.3–29.0 | within 5% ✓ |
jit(grad(fpj.prox(Soft))) |
48.2 | 44.2–45.4 | within 5% ✓ |
- Tests. In test_jax.py: |
|||
- test_jit_cache: one entry per instance, also after a new shape. Equal static arrays share a class entry; different ones do not. Results and the gradient are bitwise those of the NumPy API. |
|||
- test_jit_cache_shapes: spectral operators on three shapes, and the device TV on two. |
P10. Keep the derivative structure only when a derivative can follow (PyTorch and JAX forward)¶
- Kind: speed · Severity: medium · Target: 1.0.x · Effort: S · Depends on: P1 (touches
_Prox.forward)
Problem. Spec.run (_interop.py:67) always calls the prox with keep=True. That happens under
torch.no_grad(), when no input requires grad, and in every JAX forward callback, although JAX's
backward recomputes the prox anyway. For several operators keeping the structure costs 2–3.5×.
Setup: x = torch.randn(64, 10000, dtype=torch.float64) (its .numpy() for the NumPy column),
γ = 0.8, threads=1; operators L1Ball(1.0), Simplex(1.0), Huber(0.4, 0.5), NonNeg():
| Operator | fpt.prox under no_grad |
NumPy, structure not kept |
|---|---|---|
| L1Ball | 930.5 µs | 265.4 µs |
| Simplex | 404.5 µs | 180.2 µs |
| Huber | 250.5 µs | 116.5 µs |
| NonNeg | 177.7 µs | 66.6 µs |
The JAX jitted forward for Simplex (the same shape, radius 1, γ = 1) takes 615 µs. The NumPy call without the structure takes 187 µs, and with it 402 µs.
Design.
Spec.run(op, x, step, keep)returns(z, ctx)whenkeepand(z, None)otherwise.-
torch
_apply, beforeFunction.apply, withts = [x, step if tensor, *values]:keep = (torch.is_grad_enabled() and any(t.requires_grad for t in ts)) \ or any(fwAD.unpack_dual(t).tangent is not None for t in ts)
ctx.needs_input_grad cannot be used alone: in torch 2.14 it stays True under torch.no_grad()
for an input that requires grad. unpack_dual costs 0.11 µs per tensor and returns a None tangent
outside a dual level.
keep travels to the forward through spec. With keep False, ctx.st.fctx is None; no backward
or jvp can run in that case.
- _SpectralProx.forward: the inner op.op.prox(..., keep=keep).
- JAX: the forward callbacks of _function and _spectral run with keep=False. The backward
callbacks are unchanged (keep=True).
- Rejected: deciding keep inside forward alone, where grad mode is always off.
Constraints. Results are identical with and without keep (the same kernel). Less allocation.
No ABI change. No departure.
Tests. tests/python/test_torch.py:
- Under
no_grad, and for an input that does not require grad, the forward's context is None (check through a spy onSpec.run), and the result is bitwise equal. - Forward AD with a dual input still matches
op.jvp.
tests/python/test_jax.py: forward results are bitwise equal, and grad is unchanged.
Acceptance. Run uv run --group interop python bench/bench_python.py. The targets are for
Apple M5.
torch_wrappers section (defined in P11), lines fpt.prox(<Op>) no_grad 64x1e4:
| Operator | Before | Target |
|---|---|---|
| L1Ball | 930 µs | ≤ 300 µs |
| Simplex | 404 µs | ≤ 200 µs |
| Huber | 250 µs | ≤ 135 µs |
| NonNeg | 178 µs | ≤ 80 µs |
jax_wrappers section (P9):
fpj.prox(Simplex) jit 64x1e4: 615 → ≤ 420 µs.jit(grad(fpj.prox(Simplex))) 64x1e4: unchanged within 5%.
As implemented. As designed.
- PyTorch. _apply sets spec.keep; _Prox.forward and _SpectralProx.forward read it.
- JAX. The forward callbacks of _function and _spectral pass keep=False.
- Spec.run no longer calls steps(): its callers pass the step already converted (P11 item 5).
- Measured (µs; on this machine the NumPy reference lines take 177, 242, 132 and 81 µs):
| Line | Before | After | Target |
|---|---|---|---|
fpt.prox(L1Ball) no_grad 64x1e4 |
841 | 243–249 | ≤ 300 ✓ |
fpt.prox(Simplex) no_grad 64x1e4 |
381 | 179–184 | ≤ 200 ✓ |
fpt.prox(Huber) no_grad 64x1e4 |
174 | 136–138 | ≤ 135 |
fpt.prox(NonNeg) no_grad 64x1e4 |
163 | 84–87 | ≤ 80 |
fpj.prox(Simplex) jit 64x1e4 |
556 | 336–343 | ≤ 420 ✓ |
jit(grad(fpj.prox(Simplex))) 64x1e4 |
1625 | 1379 | unchanged |
- Huber and NonNeg. Each stays within 6 µs of its NumPy call, which is slower here than on the audit's machine (132 against 116.5 µs, 81 against 66.6). The targets were set from those faster reference numbers.
- The Simplex grad is 15% faster, not unchanged: its backward now asks for no step sum (P11 item 7).
- Tests.
test_structure_only_when_a_derivative_can_follow(test_torch.py), with a spy onSpec.run: no structure underno_grador for an input without grad, the structure for an input that requires grad and for a dual input, and the bits ofprox,vjpandjvp.test_forward_without_structure_and_grads(test_jax.py): the forward is bitwiseprox, and the gradient bitwisevjp.
P11. Trim per-call Python overhead¶
- Kind: speed · Severity: low · Target: 1.0.x · Effort: S · Depends on: P3 (the lock adds ~0.05 µs) and P2 (adds 0.08 µs); the targets include both
Problem. On fp.Soft(0.4) with n = 100, γ = 0.8, torch.set_num_threads(1) for the torch line, and
the component costs from timeit of each helper in isolation:
| Call | Time | Direct _C call |
C kernel |
|---|---|---|---|
op.prox(x, 0.8, out=z) |
1.05 µs | 0.34 µs | 0.05 µs |
Soft.vjp |
2.0 µs | 0.585 µs | |
fpt.prox (no grad) |
7.4 µs |
For comparison, NumPy op.prox is 1.1 µs and torch-native soft forward plus backward is 16 µs.
Design. Each item gives its measured cost.
_steps(__init__.py:227):if type(step) is float or type(step) is int: return float(step), None, beforenp.ndim.np.ndim(0.8)costs 0.254 µs.boolkeeps the slow path.- Compute
_batch_shapelazily: only_steps' array branch needs it (0.093 µs). Pass(shape, axis)and validate the axis withoperator.indexcheaply on the scalar path. The full range check stays in C, with P7's prefix. Soft._fields,L1Box._fields,Lq._fields: precomputeself._w0 = float(w[0]) if w.size == 1, and computelamasself._w0 * float(gp[0]) / self.lam. This replaces the one-element matmul;_fieldscosts 0.475 µs.Grads.__init__: replace thesetattrloop withself.__dict__.update(fields). Grads currently costs 0.609 µs._interop.Spec.check_names: return early whennamesis empty, and cache_derived(op)on the instance (op.__dict__["_derived_names"]). It costs 0.69 µs per forward today.- torch
_stepandSpec.runboth callsteps(). Normalise once (0.25 µs). - torch
_numpy(torch.py:87):t = t.detach(), thenif t.device.type != "cpu": t = t.cpu(), thenif t.is_neg(): t = t.resolve_neg(), thent.numpy().resolve_conjis never needed after the dtype check. 0.59 → ~0.36 µs per tensor, for x, each value, and g. - The torch and JAX backward passes request the step and parameter gradients from C only when an
input needs them (
ctx.needs_input_grad; in JAX, when the step or a parameter is traced): pass NULL forggammaandgparamsotherwise. With C5 this takes the elementwise vjp from the all-sums cost (288 µs for soft at 256 × 1000) to the gradient-only cost (49 µs).Operator.vjpin NumPy keeps returning every gradient.
Constraints. Pure Python, no behaviour change, no departure.
Tests. The existing suite. A micro-benchmark assertion is not added, since timings are flaky in CI.
Benchmark. Two additions to bench/bench_python.py.
overhead(). Three lines on the existing x (n = 100) and out, with soft = fp.Soft(0.4) and
_, sc = soft.prox(x, 0.8, keep=True) built outside the timing:
| Line | Call |
|---|---|
Soft().prox(x, out=out) |
soft.prox(x, 0.8, out=out) |
Soft vjp(ctx, x) |
soft.vjp(sc, x) |
Soft jvp(ctx, x) |
soft.jvp(sc, x) |
New section torch_wrappers(), called from __main__ before jax_wrappers.
- It starts with
try: import torch; import fastprox.torch as fptandexcept ImportError: print("PyTorch wrappers: torch not installed, skipped"); return. - It calls
torch.set_num_threads(1). - On
xt = torch.from_numpy(np.random.default_rng(0).normal(size=100))andxr = xt.clone().requires_grad_(True):
| Line | Call |
|---|---|
fpt.prox(Soft) no grad |
fpt.prox(soft, xt, 0.8) |
fpt.prox(Soft) fwd+bwd |
fpt.prox(soft, xr, 0.8).sum().backward() |
- On
X = torch.from_numpy(rng.normal(size=(64, 10000))), undertorch.no_grad()withthreads=1, forL1Ball(1.0),Simplex(1.0),Huber(0.4, 0.5)andNonNeg(): fpt.prox(<Op>) no_grad 64x1e4:fpt.prox(op, X, 0.8, threads=1).<Op>().prox 64x1e4: the NumPy reference,op.prox(X.numpy(), 0.8, threads=1).
Acceptance. Run uv run --group interop python bench/bench_python.py (plain
uv run python bench/bench_python.py runs the NumPy lines alone). The targets are for Apple M5:
| Section and line | Today | Target |
|---|---|---|
overhead: Soft().prox(x, out=out) |
1.05 µs | ≤ 0.80 µs (with P2 and P3) |
overhead: Soft vjp(ctx, x) |
2.0 µs | ≤ 1.4 µs |
torch_wrappers: fpt.prox(Soft) no grad |
7.4 µs | ≤ 6.0 µs |
The existing overhead lines (fp.soft_threshold(x) 0.67 µs, fp.project_simplex(x) 0.72 µs,
fp.tv1d(x) 0.92 µs, TV1D().prox(x, out=out) 1.26 µs on Apple M5) do not regress, and neither do
the throughput and scaling sections.
As implemented. Items 1–7 as designed, with these details:
- Item 2. _steps(step, conj, op, shape, axis) computes the batch shape only for an array step.
- Item 3. The base Operator._fields indexes a scalar entry instead of slicing it (0.23 → 0.18 µs for Soft).
- Item 7.
- Operator.vjp calls Operator._vjp(..., want_gamma, want_params); the binding's _C.vjp gains want_gamma, which passes NULL to fp_vjp.
- The backwards ask for the step's sum only for a scalar step that needs a gradient. In JAX that means a traced step (a Tracer). A per-row step's gradient comes from a jvp.
- They ask for the parameters' sums only when a parameter is differentiable. Grads then has no fields.
- _deriv_info takes the warning's stack level, so an AccuracyWarning still points at the caller's line through vjp.
- Measured (Apple M5 Pro, three alternating runs, µs):
| Line | Before | After | Target |
|---|---|---|---|
overhead: Soft().prox(x, out=out) |
1.08–1.12 | 0.71–0.73 | ≤ 0.80 ✓ |
overhead: Soft vjp(ctx, x) |
1.71–1.79 | 1.42–1.43 | ≤ 1.4 |
overhead: TV1D().prox(x, out=out) |
1.33–1.38 | 0.94–0.98 | no regression ✓ |
torch_wrappers: fpt.prox(Soft) no grad |
7.8 | 5.4–5.6 | ≤ 6.0 ✓ |
torch_wrappers: fpt.prox(Soft) fwd+bwd |
21.3 | 18.2–18.4 | — |
- Open:
Soft vjpis 20% faster, against a target of 30% (2.0 → 1.4 on the audit's machine). The C call takes 0.43 µs of the 1.42.Gradsand its fields take 0.29, and the input and output arrays about 0.3. - The
fp.soft_threshold,fp.project_simplexandfp.tv1dlines are unchanged: 1.10–1.12, 1.19–1.25 and 1.38–1.49 µs before, against 1.10–1.11, 1.19–1.20 and 1.38–1.40 after, in three alternating runs. - Tests. The existing suite.
test_deriv_keywords.pynow spies onOperator._vjp, which bothvjpand the wrappers' backwards call.
P12. Free-threaded CPython (3.13t, 3.14t)¶
- Kind: docs · Severity: low · Target: 1.1 (wheels), 1.0.0 (a docs line) · Effort: M · Depends on: P3 (required), P4
Problem. Support is not claimed (pyproject.toml classifiers) and was not tested in this audit:
no free-threaded interpreter was built.
- The extension is built for the stable ABI:
nanobind_add_module(_fastprox STABLE_ABI NB_STATIC ...)inbindings/python/CMakeLists.txt:10, withwheel.py-api = "cp312". It has noFREE_THREADEDflag. - Free-threaded interpreters do not load abi3 extensions (the limited API is unavailable there before 3.15), so the published wheels cannot be installed. pip falls back to the sdist.
- An sdist build without
FREE_THREADEDproduces a module that does not declarePy_MOD_GIL_NOT_USED. Expected behaviour, unverified: CPython re-enables the GIL at import with a RuntimeWarning. The package then works with the GIL on.
Design. Current position, for docs/api.md (Installation): "Free-threaded CPython builds are not supported yet; built from source, the extension re-enables the GIL." Support would need:
FREE_THREADEDinnanobind_add_module. It is ignored by GIL builds. Check that nanobind and scikit-build-core dropSTABLE_ABIandpy-apifor free-threaded targets and emitcp313t/cp314twheels.- Python-level shared state:
Contextmutation needs the P3 lock. Without the GIL the attribute writes (info,step,conj,deriv) also race.- Operators are immutable after construction.
- The P9 caches tolerate races (a duplicate compile at worst).
_branchis covered by P3.- The C side is already exercised by concurrent callers with the GIL released: the thread pool "runs
concurrent callers serially" (decision of 2026-10-15), and 8 Python threads scaled 7.06×
(
TV1D(0.6)on 32 arrays of (64, 20000),threads=1per call, 8 threads against 1). - CI:
- cibuildwheel with free-threading enabled for cp313t and cp314t.
- A test job with
PYTHON_GIL=0running the suite, the P3 concurrency test, and that scaling check as a test. - numpy ≥ 2.1 (free-threaded wheels).
- The interop tests skip where torch or JAX have no free-threaded wheels.
- The classifier
Programming Language :: Python :: Free Threading :: 2 - Betaonce the job is green.
Constraints. No change to C, the ABI or determinism. ROADMAP 1.1 gets the item "Free-threaded CPython wheels (cp313t/cp314t)". No departure.
Tests. The CI job in item 4.
As implemented (1.0.0 part). The line is in the README's Install section and at the top of the Python section of docs/api.md (which has no Installation section). The expected behaviour is now checked: on CPython 3.14.7t, uv pip install . builds the extension, and the import enables the GIL with a RuntimeWarning, after which the package works. The 1.1 item is in ROADMAP.md under "1.x — candidates".
Acceptance. On cp314t, pip install fastprox installs a wheel, import fastprox leaves
sys._is_gil_enabled() False, and the suite passes under PYTHON_GIL=0.