Abhijeet Sinha

Projects

Matryoshka Transcoders

Automatically finding, and describing in plain English, the ways image generators break physics, such as extra fingers, warped text and impossible poses.

In one sentence: we train a classifier to spot physically implausible images, split what it has learned into many small interpretable features, and let a multimodal language model name each one. The result is a catalogue of how image generators fail.

The problem

Modern image generators produce pictures that look convincing at first glance but break basic physical rules: a hand with six fingers, a tennis player whose arm bends the wrong way, poster text that is not quite letters. These mistakes matter in real applications, yet the standard metrics miss them. Those metrics check whether an image matches its caption or whether a batch of images looks realistic overall. Even strong multimodal models struggle to notice these errors reliably when they look at an image as a whole.

What’s missing is a way to find the specific, recurring kinds of mistakes automatically and describe them in words, so model builders know what to fix.

The approach

Figure 1 · From labelled images to named failure modes

1 · Humans labelgenerated imagesplausible or not2 · A classifierlearns the difference(frozen CLIP + head)3 · Matryoshkatranscoder splits itinto sparse features4 · A multimodal LLMreads top imagesand names each oneExample outputFeature 243: "malformed, unreadable text on posters and signs"
The classifier already "knows" when an image looks physically wrong, but that knowledge is tangled up inside it. The transcoder untangles it into separate features, and a language model gives each feature a plain-English name. (Feature 243 is one of the paper's real examples.)
  1. Label. Two annotators marked 3,410 generated images as physically plausible or not. We added 5,000 real photos from MS-COCO as extra plausible examples.
  2. Classify. A frozen CLIP image encoder with a small two-layer head learns to tell the two classes apart. Its internal activations now carry the information about physical errors, but tangled together.
  3. Untangle. A transcoder is a small network that rebuilds one layer’s activations from the layer before it, using only a few active features at a time. That sparsity pushes each feature to stand for one recognisable thing.
  4. Describe. For each feature we collect the images that activate it most and ask a multimodal model two questions: what do these images have in common, and is it a physical-plausibility error?

Why “Matryoshka”?

Plain sparse models must choose between a few broad features and many narrow ones. A Matryoshka model trains nested dictionaries together, so you get both.

Figure 2 · The Matryoshka idea (illustrative)

One dictionary of 2048 features, trained so that every prefix works on its ownfirst 2048first 1024first 512first 256first 128coarse, common errorsfine-grained, rarer errors
Like nested Russian dolls: the first 128 features must reconstruct the classifier's signal on their own, then the first 256, and so on up to 2048. The small sets are forced to capture the broad, common failure types, and the extra features are left to add finer detail.

What we found

The discovered features are clear and easy to name. Examples from the paper include anatomical distortions in tennis scenes (feature 104), malformed text on posters (feature 243) and implausible hand-drawn styles (feature 648). Compared with standard sparse autoencoders, Matryoshka sparse autoencoders and plain transcoders, Matryoshka transcoders produce the largest share of features that are really about physical errors.

Figure 3 · How many discovered features are about physical errors

population-based relevancedescription-based relevance0%5%10%15%20%relevant features6.5%6.0%SAE9.9%5.7%Matryoshka SAE12.7%11.5%Transcoder16.0%13.0%Matryoshka Transcoder
The share of learned features that fire almost only on implausible images (population-based) and whose automatic description is judged relevant (description-based). Numbers are from Table 1 of the paper.

The features can also be reused as a benchmark. Run any generator’s images through the pipeline and count how many error features fire on each image. Across eight popular models, this ranks them clearly and also shows which kinds of error each one makes.

Figure 4 · Benchmarking eight image generators

average number of error features firing per image (lower is better)SDXL-turbo74.2SDXL-base53.1DALL-E 349.8SDXL-medium48.7SDXL-large46.8Stable Cascade43.1Kolors40.8FLUX38.0
Once the error features are found, they can be used to score any image generator. Each model produced 5,000 images. FLUX makes the fewest physical mistakes and SDXL-turbo the most (Table 2 of the paper).

An ablation on dictionary size shows the trade-off that motivates the nested design. Small dictionaries give more focused features (17.2% relevant), while large ones cover many more error types (1,896 relevant features compared with 170). Nesting several sizes captures both.

Limits and next steps

The discovered features diagnose failures but are not yet used to train them away. That is the obvious next step. Rare kinds of error can also be missed if the labelled data contains few examples of them.