Same 2 379-parameter char-LM as lm.html, same byzantine
defense, same federated text shards. The proposal shape is different:
instead of submitting (indices, values) for 6 random params,
each worker probes a coherent Gaussian direction via SPSA and submits
(seed, scalar_g) — a 20-byte step on all P params.
This is the fusion with DeComFL
that lifts the protocol's parameter ceiling from ~10K toward LLM scale.
Live sample — 100 chars seeded from "t", greedy + temperature:
SPSA per round, per worker: 4 trials × (3 forward passes + 1 lossBefore reuse) ≈ 13 forwards.
Each trial probes (L(θ+εu) − L(θ−εu)) / 2ε along a fresh Gaussian direction u
(seed → mulberry32 → Box-Muller, bit-reproducible on the server). The server picks the
worker whose probe predicts the steepest local loss drop and applies
θ ← θ − η·g·u. ε = 0.01, η = 0.005, P = 2 379.
Why this matters: a single 20-byte proposal nudges all 2 379 parameters in
a coherent direction. For the current toy model the per-round bandwidth is similar to
lm.html's flip-and-accept, but the protocol shape is identical
whether P = 2 379 or P = 3 800 000 000.