Our research starts from a single observation: the systems we use to find, retrieve, and organise information almost universally work by similarity. Given a query, find the most similar stored item. This assumption — that useful connections are similar connections — is so deeply embedded in modern AI that it's rarely questioned.
But it fails in exactly the situations where memory matters most. Stairs don't resemble a slip, yet one reliably evokes the other. A chord progression retrieves a person. The smell of sunscreen retrieves a holiday. In every case, the connection exists not because the two experiences are alike, but because they were experienced together. Temporal co-occurrence — simply being present in the same window of experience — is a universal associative signal that requires no annotation, no supervision, and no representational overlap between linked items.
We're building the theory, mechanisms, and systems that take this signal seriously.
The foundational paper. We propose a JEPA-style predictor trained on temporal co-occurrence within a continuous experience stream. The predictor learns to navigate the associative structure of an embedding space — retrieving items linked through shared experience rather than representational proximity.
On a synthetic benchmark, the predictor's top retrieval is a true temporal associate 97% of the time. It recovers associations across representational boundaries where cosine similarity scores zero. A temporal shuffle control confirms the signal is genuine co-occurrence structure, not embedding geometry. A held-out evaluation confirms anchor-specific recall: the predictor remembers what it experienced, from the perspectives at which it experienced it.
The key distinction is between retrieval (finding relevant items, where generalisation is the goal) and recall (reactivating specific associations formed by specific experience, where faithful memorisation is the goal). PAM is a recall system. It doesn't generalise to associations it never formed — and this is correct behaviour, not a failure mode.
The applied paper. We operationalise the PAM insight for a practical problem: multi-hop question answering, where a retrieval system must find passages that are associatively related through shared reasoning chains, not just similar to the query.
AAR trains a lightweight MLP (4.2M parameters) on passage co-occurrence and uses it to rerank dense retrieval results. On HotpotQA, it improves Recall@5 by 8.6 points without evaluation-set tuning, with gains concentrated on the hardest questions (+28.5 points where the dense baseline fails). On MuSiQue, it achieves +10.1 points.
Four ablations tell a consistent story: training on similar-but-not-associated pairs degrades retrieval (association ≠ similarity, confirmed). An inductive variant shows no significant improvement — the method captures corpus-specific co-occurrences, not transferable patterns. The method adds 3.7ms per query, trains in under two minutes, and requires no LLM-based indexing.
Cross-domain biological validation on gene perturbation data (Replogle K562 + STRING protein interactions) shows the same principle works beyond text: cross-boundary AUC improves from 0.534 to 0.902 at high confidence. Inductive transfer partially works in biology (+0.152), unlike text — suggesting a spectrum of association transferability from fully contingent to physically grounded.
A reliability-weighted learning mechanism where component adaptability is determined by predictive accuracy. Plasticity reactivates automatically during distribution shifts — no external schedule required.
Experimental validation confirming the core CWP mechanism functions as intended, alongside identification of boundary conditions in tightly-coupled architectures where shared parameters cause the mechanism to malfunction.
The emergence paper. We train PAM at scale on 10,000 Project Gutenberg novels (24.96 million text chunks) and show that the compression mechanism discovers hierarchical narrative concepts without supervision.
At coarse granularity (k=50), the model separates broad narrative modes — verse from prose, action from reflection. At fine granularity (k=2000), it distinguishes specific narrative techniques — "urgent private conversation" from "formal social exchange." The same concept clusters appear across authors, genres, and centuries, suggesting that narrative structure has universal regularities that temporal co-occurrence alone can recover.
Held-out novels (never seen during training) receive coherent cluster assignments through inductive inference, demonstrating that the learned concepts generalise to new material. Authorial pacing signatures emerge naturally: Tolstoy sustains single narrative modes for hundreds of chunks; Joyce cycles through modes within paragraphs.
A core prediction of the programme is that temporal co-occurrence captures useful structure in any domain where items are experienced sequentially or contextually. We've tested this in three domains.
Narrative passages in novels. PAM learns which passages serve similar structural functions across thousands of books. AAR learns which passages co-occur as supporting facts for multi-hop questions.
Gene perturbation profiles. The same contrastive architecture, trained on STRING protein-protein interactions instead of passage co-occurrence, learns to identify interacting gene pairs that expression similarity misses entirely. Where expression cosine similarity is near chance (0.534), the learned association function achieves 0.902 AUC.
Identified as the next validation domain. Citation structure in legal corpora provides a natural co-occurrence signal analogous to passage co-occurrence in QA datasets.
The consistent finding across domains: association and similarity produce different — often opposite — rankings, and association captures relationships that similarity systematically misses.