representation-geometry

Trajectory Geometry of Transformer Representations Across Layers

[EMNLP 2026 Findings] Decomposing Transformer updates into parallel & perpendicular subspaces for geometric probing, compression, and training dynamics.

Prosthetic cognition architecture for AI agents. Deterministic scaffolding over probabilistic reasoning.

Code, results and paper for "The geometry of single-cell foundation models: what they inherit, what they add, and what shapes it"

Corpus operators recover different structures, and task-aligned conditionals predict held-out model behaviour — with musical keys as the instrument. Code, results and audit trail for the paper.

Mechanistic interpretability of multilingual reasoning in transformers. 170+ causal intervention experiments across 4 model families.

Effective rank, RankMe, E1, CKA and anisotropy on transformer hidden states are determined by one direction. The exact identity, and the attention sink behind it.

A 2-D map of GPT-2's token embeddings you can poke at. Pick two axes, project the vocabulary, and see where analogies, cyclic features, and hubness show up (and where the 2-D view is lying to you). Companion to a writeup on which "cyclic" concepts actually form circles.

Mechanistic interpretability of transformer hallucinations via attention flow, residual stream geometry, and head-level attribution analysis.

Visualizing Modern LLM Mechanics, Loss Landscapes & HPC Topologies

Exploring the internal representations of open source LLMs, seeing how they encode different concepts, such as meaning, languages, countries...

Code for 'Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance' (arXiv:2406.12159)

Personal learning notebook on latent space engineering, representation geometry, and exploratory research notes.

Reproducibility package for "Context Is King: How In-Context Specification Shapes the Geometry of Concepts" — code, cached data, and interactive 3D explorers.

Code for 'Reliable Measures of Spread in High Dimensional Latent Spaces' (ICML 2023)

Com la tokenització fractura la morfologia catalana i si una segmentació conscient dels morfemes recupera la geometria. Provat en 3 llengües indoeuropees (català, castellà, anglès): el català es fragmenta ~1,7× més que l'anglès; forçar el tall morfèmic recupera la composicionalitat (robust a portadora i replicat en castellà).

An empirical and geometric analysis of Neural Collapse under different optimizers on CIFAR-10

Code for paper "The Geometry of Contextual Relations: Language Models Address Facts by Order of Mention"