---
title: "I tried to animate embeddings instead of flattening them"
date: 2026-07-12
description: "A compact experiment animates raw, PCA and grand-tour views of embeddings, then changes the basis without changing cosine neighbours."
tags:
  - "embeddings"
  - "data-visualisation"
  - "machine-learning"
  - "ai-experiments"
---

I have been playing with embeddings much more over the last two or three months while learning about AI. I first tried using them to modulate sound. I also used them in a code-discovery project, embedding code and looking for similar code by matching the vectors.

Both projects led me towards PCA and UMAP projections, which I thought were pretty cool. I do not really know where the next idea came from. It just popped into my head: “Huh, it'd be cool to try and animate embeddings rather than just flatten them.”

I also thought, “Oh, this is just not going to work.”

It worked, which surprised me.

## The idea already had a name

My first version moved through pairs of coordinates. Instead of choosing one flat projection and leaving it there, it would show dimensions 1 and 2, move towards dimensions 2 and 3, then keep going.

I later discovered that this sits inside an established family of visualisations called tours. Daniel Asimov described the [grand tour in 1985](https://doi.org/10.1137/0906011) as a way to move through two-dimensional projections of multidimensional data. The [`tourr` project](https://ggobi.github.io/tourr/articles/intro.html) gives the more precise names: cycling through axis-aligned views is a little tour, while a grand tour moves through more general projection planes.

So I had not invented a new visualisation technique. I had kind of independently arrived at a known idea, or at least the entrance to one. That was probably the coolest part. It made me feel like I was starting to understand embeddings well enough to reach towards something that was already a real, useful idea elsewhere.

There is a lot more work in this area than my small experiment. [Distill used the grand tour to inspect neural-network activations](https://distill.pub/2020/grand-tour/), and the recent [dtour project](https://arxiv.org/abs/2605.04306) builds a much more complete browser interface for steering through high-dimensional data and embeddings. Mine is not a replacement for either. I reached a much smaller version first, then found the wider field around it.

```interactive
id: embedding-tours
title: "Embedding tours"
height: 900
mobile_height: 900
```

The instrument has a deterministic eight-dimensional synthetic dataset and 180 short phrases embedded into 384 dimensions with MiniLM. You can cycle through raw coordinate pairs, move through pairs of principal components, or use a seeded grand tour through general projection planes that are not aligned to the raw axes.

Those options are meaningfully different. The raw tour moves through coordinate pairs. PCA puts the highest-variance directions first, although that does not make them the true semantic axes. The grand tour is not limited to axis-aligned pairs. They do not feel like three labels on the same view. They are all different and cool.

Every frame is still only a two-dimensional projection. Animating it does not recover all of the information that was lost when hundreds of dimensions were put on a flat screen. It gives you more partial views.

It is also just really satisfying to watch the structure move.

## The basis switch

The raw-coordinate mode has a switch between the original basis and an orthogonally transformed one. When you flip it, the projection can look substantially different. The individual coordinates have changed.

The relationships measured by cosine similarity have not. A shared orthogonal change of basis preserves dot products, lengths, distances, and cosines. The experiment keeps all five cosine neighbours for every phrase, with the measured numerical drift below `2e-15`. This is the narrow invariance behind the comparison, and it is also why rotations matter when discussing the [interpretability of embedding coordinates](https://arxiv.org/abs/1707.04662).

That does not mean every possible transformation leaves an embedding unchanged, or that individual axes can never be useful. It means the cosine-neighbour structure does not uniquely privilege the raw coordinates I happened to receive from the model. A very different-looking coordinate tour can represent the same neighbour geometry.

## It stayed small

Often I build tools and there might be a nugget of satisfaction hidden away in them, but most of it is just bloated. This does not feel like that. It is brief and does what it does pretty well. The two datasets, three projection modes, basis comparison, selection and playback controls all change the experience in ways I can notice.

I do not expect a researcher to arrive here and discover a new technique. Maybe it will be interesting to someone like me who is still learning and finds embeddings confusing. Maybe it will help the idea make slightly more sense.

Maybe this won't. Maybe it is just a cool experiment. I think it is worth sharing.