Oscillator attention.

Generic Kuramoto-style networks admit many equilibria. For some computations this multiplicity is the point: associative memory exploits exponentially many stable phase-locked configurations; Ising-machine optimizers encode combinatorial solutions as distinct fixed points; general oscillator neural networks draw on the same richness. For attention, however, multiple equilibria would be a bug: the same query in the same context could produce different attention patterns depending on initialization, breaking determinism.

The fixed-query design removes the ambiguity. Anchors, the orange squares in the video above, are external forcing terms rather than free oscillators, so each free oscillator evolves independently of the others: it descends a potential set by the weighted anchor sum alone. On the sphere that flow has exactly two equilibria — the settled state, which is the closed-form normalized anchor sum, and its antipode. The settled state is asymptotically stable and the antipode is unstable, so every trajectory that does not start exactly on the antipode converges to the same point. Attractivity is almost-global rather than global, and that distinction is the honest one: the exception is a single point, but it is not empty.

This matters because the guarantee is what transfers to hardware. It holds for every physical realization of the flow, which is what justifies training on the algebraic fixed point and then deploying the physics. In software the settled state is computed in closed form, so the two ways finite-time convergence can fail — a near-cancelling anchor sum, or an initialization near the antipode — bear on physical deployment only, and both are controlled by the oscillator dimension.

The video below makes the property visible. One query, three random initial conditions for the active oscillator, one settled state. The three colored trails fan out across the unit circle and funnel into a single point. The attention row and the candidate-probability bars, both read out from the settled state, collapse onto the same values whichever trail was taken.

The objective is not to replace softmax in software. It is to show that a mechanism a physical oscillator array can execute is accurate enough to be worth building, so softmax is used as a calibration baseline rather than a target to beat. We evaluate on keyword spotting (KWS), subject-verb agreement (SVA), and causal language modeling (TinyStories and WikiText-2).

At the minimal hardware configuration — oscillator dimension d_osc = 2 — the two mechanisms are at parity on both bidirectional tasks. On KWS the comparison has to be made under matched positional information: the anchors are learned per absolute position, so they act as a built-in positional encoding of standard strength, and an oscillator model compared against a baseline with no positional encoding is being flattered by that. Under matched encoding the two land within a few tenths of a point of each other. On SVA the accuracies also match; what differs is reliability. Training runs that fail outright are markedly rarer for the oscillator, 3 seeds in 50 against 12 in 50 for softmax, which is where the wide error bars in the right-hand panel come from.

Causal language modeling is the harder setting, and softmax retains a real advantage there. The useful result is that the gap is not stubborn: it decays as a power law in the oscillator dimension, and the fitted relation predicts a configuration it was never shown. Fits over d_osc from 2 to 32 were used to forecast d_osc = 64; the held-out runs land inside the 95% prediction interval on both datasets. The dimension of the substrate, not the mechanism, is what bounds accuracy here.