Do two models given
opposite halves of a protein
learn the same biology?

If sequence and shape are two views of one biology, two models should agree

Everything a protein does is supposed to follow from its shape, and its shape from its sequence. That chain of reasoning is the foundation of structural biology, and it implies something testable about machine models: a model that has only ever read sequences and a model that has only ever read coordinates are looking at two views of the same object. Given enough of each, they should arrive at the same internal picture of proteins. This is the protein version of a broader claim — that large models trained on different data converge on a shared representation of the world (Huh et al., 2024).

It matters practically. It is the assumption behind using one model's output in place of the other's, behind combining them, and behind treating either as a general-purpose description of a protein. If it holds, the two are interchangeable in ways that would save a great deal of work. If it holds only partly, it matters exactly where the limit falls.

One pair is the experiment; the rest set the scale

Everything detailed below — the maps, the layer grids, the stitching, the property tests — is ESM-2 35M (sequence) against the ProteinMPNN encoder (structure). They are matched in the way that matters here: both are small, both produce one description per residue for the same residues, and they read opposite halves of a protein.

Three ways to ask “do they agree?”

Two taxonomists can use the same traits to organise species and still shelve them differently. These three measures separate exactly that. CKA is centred kernel alignment (Kornblith et al., 2019); SVCCA is singular-vector canonical correlation analysis (Raghu et al., 2017); mutual k-NN compares k nearest neighbours, as used by Huh et al. (2024).

Calibrating each measure on scrambled residues

No similarity score means anything on its own. Scramble which residue is which and every correspondence between the two models is destroyed; whatever a measure still reports is what it returns for unrelated data. For the main pair, ESM-2 35M against ProteinMPNN: same pattern of resemblance reports 0.258, of which 1% is null; same main directions reports 0.415, of which 69% is null; same neighbours reports 0.029, of which 8% is null. Two high-dimensional clouds of numbers always share some direction, whether or not they are related, so most of the headline “same main directions” score is noise floor. The other two measures are nearly clean.

It gets worse when models of different sizes are compared, because the scrambled score grows with the number of dimensions. The bigger model looks far more convergent — 0.598 against 0.408 — and is not: read against its own scrambled score it is identical to the small one. Scores like this cannot be compared between models of different widths without a null. This is the same width effect that Gröger, Wen & Brbić (2026) report for vision–language models, found here independently in proteins.

The strongest agreement between the two families was the amino-acid code both are trained to output

Both families are trained to output the amino acid at each position — the sequence model by filling in masked residues, the structure model by saying which amino acids would fold into a given backbone. So two models can look like they agree simply by both knowing that position 47 holds a leucine, without sharing any understanding of proteins at all.

This is not hypothetical. Measured by CKA against ProteinMPNN's last encoder layer, ESM-2 agrees with the structure model most at its very first layer, the embedding layer: a lookup table that gives each amino acid a fixed vector, before any context or learned processing. That layer is nothing but the amino-acid code. Compared with a bare label naming the amino acid, it scores 1.000 by SVCCA and 0.916 by mutual k-NN, so agreement measured there is the shared training target, not shared biology. Deeper in the model, where the learned biology should be, CKA with the structure model falls, from 0.270 at the embedding layer to 0.110 at block 7, and rises again toward the output, 0.192 at block 12, where the model returns to predicting amino acids. The layers doing real work agree with the structure model less than the bare amino-acid code does.

Removing the answer key

Each model turns every residue into a list of numbers. To remove the shared answer key, take every leucine in the dataset, average its description, and subtract that average from each individual leucine — then repeat for all twenty amino acids. What is left is how this leucine differs from a typical leucine: its surroundings, its role. If a model knew only the amino acid, subtracting would leave exactly zero and its apparent agreement would vanish entirely; that is the intended behaviour. If a model knows real biology — where the residue sits, what surrounds it, whether it is buried, what it does — all of that survives untouched.

Two checks show the subtraction does what it should. The same model trained twice — agreement that is certainly real — barely moves: 0.849 → 0.849. An untrained sequence network against a trained structure model — agreement that cannot be about biology, because one side has learned nothing — scores a respectable-looking 0.205 as measured and collapses to 0.038 once the answer key is gone. Real agreement is left standing; manufactured agreement is removed. Every result below is reported both ways: as measured, and with the answer key removed.

Layer by layer, the subtraction redraws the first chart. ESM-2's embedding layer drops from 0.263 to nothing: it held only the amino-acid code. The early blocks keep a small part of their agreement (0.257 → 0.067 at block 1), and what remains grows with depth, highest at block 11 (0.257 → 0.137). Before the subtraction the input layer led; after it, the agreement that is left sits where ESM-2 has done the most processing.

With the answer key removed, every sequence–structure pair still sits well above the floor — and well below two models that share an input

A score of 0.13 means nothing on its own until you know what the same measurement gives for two models that certainly agree and for two that certainly do not. So every pair is read against a ladder. At the top, the same model trained twice: 0.849, unchanged by the subtraction. Next, two models sharing an input: two sequence models (CARP-38M × ESM-2 35M) 0.605 and two structure models (ESM-IF1 × ProteinMPNN) 0.468 after the subtraction (0.693 and 0.522 as measured). At the bottom, the floors: an untrained sequence network against a trained structure model 0.038, the reverse 0.122, two untrained networks 0.022.

Every sequence-against-structure pair lands between those rungs. The main pair, ESM-2 35M against ProteinMPNN, goes from 0.258 as measured to 0.131 with the answer key removed; the other five cross-modal pairs land between 0.111 and 0.245 after removal, the largest pair highest. All six sit clearly above every floor and clearly below the same-input rungs. Same main directions barely moves for any pair (0.415 → 0.397 for the main pair) — as the calibration showed, most of that score was noise floor to begin with, so there was little answer key in it to remove.

They do not arrange proteins the same way

Sharing axes is weaker than sharing a map. The neighbour measure asks, for each residue, how many of its ten closest neighbours in one model are also among its ten closest in the other. Two copies of the same model agree on 0.796 of each other's neighbours after the subtraction (0.850 as measured); two sequence models on 0.471; two structure models on 0.107. No sequence–structure pair exceeds 0.06, and the main pair, ESM-2 35M against ProteinMPNN, scores 0.030. The two families line up along some of the same directions, but they do not put the same residues next to each other — they are not two drawings of one map.

Two sequence models are the one pair whose neighbour agreement rises after the subtraction (0.246 → 0.471): removing the twenty islands stops amino-acid identity from dominating the neighbour lists, so neighbours are compared within an amino acid. Cross-modal pairs do not move (0.029 → 0.030), which supports the reading that the subtraction removes the islands and leaves the biology intact.

Agreement is highest where the sequence model has learned least

The same pair of models, every layer of one against every layer of the other, as measured. Read along the sequence model's depth: same pattern of resemblance is highest at the embedding layer (0.270 against the structure model's last encoder layer), falls through the middle blocks to 0.110 at block 7, climbs back to 0.256 at block 11 and dips to 0.192 at the output. Same main directions is nearly flat (0.30–0.40) and mostly null. Same neighbours never leaves the floor (0.009–0.024). Read along the structure model's depth: its last encoder layer is the most similar to every sequence layer, on every measure. The three panels disagree because the measures do.

The structure model's description does not survive the sequence model's layers intact

Similar is not the same as interchangeable. Stitching — taking one model's description of a residue and feeding it into the other model's remaining layers — tests the stronger claim directly: take the structure model's description of each residue, pass it through a single trained translation step into the sequence model at some layer, and let the sequence model's own remaining layers finish the job. Every entry point was tried — 36 combinations of structure-model layer and entry point — and each stitched model was scored on three properties: 108 cases, each repeated on 5 independent samples of proteins. The comparison that settles it is against the structure model on its own: scoring against the sequence model would flatter stitching, because the structure model is simply better at these properties to begin with (Lenc & Vedaldi, 2015; Bansal et al., 2021).

Passing the structure model's work through the sequence model's trained layers loses part of it. In 77 of 108 cases the stitched model is clearly worse than the structure model alone, a difference that holds up across all 5 repeats; in 28 there is no clear difference, and in 3 it is better. The losses are not spread evenly: every case that enters in the first 4 layers is worse, and the cases with no clear difference sit at the deep entry points, where few of the sequence model's layers are left to act.

What comes out is in between. Untrained layers pass the information through unchanged (0.828 → 0.833 for local shape, 0.879 → 0.878 for buried or exposed, 0.767 → 0.766 for solvent accessibility). Trained layers keep more than the sequence model knows on its own and less than the structure model supplied: for solvent accessibility, 0.598, against 0.324 for the sequence model alone and 0.767 for the structure model alone. The layers are tuned to inputs a structure model never produces.

Two checks bound the result. The translation step is a single linear map, and on held-out proteins it captures only 0.02–0.17 of the variance (mean 0.067). A small neural network in its place fits slightly better (mean 0.112) and changes nothing: 10 of 12 cases are still clearly worse. And the same test can pass. Fed into ESM-2 from CARP, a second sequence model, the linear map fits about 5 times better (mean 0.324) and 24 of 36 cases show no clear loss. CARP starts at roughly ESM-2's level on these properties, which makes a no-loss result easier to reach; the structure model carries information the sequence model lacks, and that is what is lost. Folding trunks that share both input and target have likewise been shown to be functionally interchangeable (Lu et al., 2026).

The other direction: ESM-2 into ProteinMPNN

The test also runs the other way. ESM-2's description of each residue, through the same kind of translation step, replaces ProteinMPNN's own after its first, second or third encoder layer, and ProteinMPNN's remaining layers finish the job: 39 combinations of ESM-2 layer and entry point, each repeated on 5 independent samples of proteins. One thing differs from the first direction. ProteinMPNN passes messages between each residue and its nearest neighbours in space, and that neighbour graph comes from the coordinates, so its remaining layers always have the geometry whatever is fed in. Two references are therefore needed: ESM-2 on its own, and ProteinMPNN's remaining layers given nothing but the geometry.

A fair test needs a property ESM-2 reads at least as well as ProteinMPNN. At the level of single residues only one qualifies here: whether a residue contacts a ligand, which ESM-2 alone scores 0.602 and ProteinMPNN alone 0.595. Passed through ProteinMPNN's layers, ESM-2's description of it scores 0.568: clearly below ESM-2 alone in 27 of 39 cases, and clearly above the geometry on its own (0.531) in 32 of 39. Part of what ESM-2 knows survives and part is lost — the same in-between result as in the first direction.

The structural properties show how the two sources combine. For solvent accessibility the stitched model scores 0.676: above ESM-2 alone (0.329) and above the geometry alone (0.549), so ESM-2's description adds to what the coordinates give, but below ProteinMPNN alone (0.835). Active sites and sites of chemical modification are too rare among residues to score here — the modification sites sit at chance level in every condition — so their cases are not counted either way.

Each model reads best what it was shown — except fold topology

Both models were tested on properties neither was trained to predict, with a probe on each layer and the best layer reported. The split is clean, and it runs along the boundary between geometry and biology. Everything geometric — solvent accessibility, buried or exposed, local shape, flexibility (B-factor) — is read better from the structure model: 0.816 against 0.420 for solvent accessibility (R²), 0.892 against 0.761 for buried or exposed, 0.887 against 0.786 for local shape, 0.276 against 0.180 for flexibility (R²). Everything functional and evolutionary is read better from the sequence model: enzyme class 0.550 against 0.318, protein class 0.507 against 0.266, kingdom 0.688 against 0.387, each with the best amino-acid-counting baseline well below the sequence model (0.263, 0.203, 0.580).

The exception is fold topology. It is a shape category — and the model that has never seen a shape predicts it better than the model that has seen nothing else: 0.685 against 0.489, with a counting baseline of 0.299. Evolutionary constraint, visible in sequence, carries information about fold at least as directly as this structure model's view of geometry does.

Scores are macro-averaged F1 unless marked R²; intervals are the spread across five independent refits. The sequence model has 13 layers to pick its best from and the structure model 3, which favours sequence — the gaps are far larger than that can explain, but the absolute numbers are optimistic for both. Six further properties were tested and are excluded from these claims because their intervals overlap or their baselines do nothing but guess the commonest answer; they are shown in the full battery below.

Geometry is handed to the structure model; the sequence model has to build it

Left is each model's input, right is its output. The structure model already knows the geometry at its first layer — it is handed the coordinates, so there is nothing to build — and reaches its ceiling almost immediately (local shape 0.887 at the first encoder layer). The sequence model has to construct that knowledge, climbing steadily through twelve blocks and never quite arriving (local shape 0.556 at the embedding layer to 0.778 at block 11). For function it is the reverse: the sequence model rises far above the amino-acid-counting baseline — fold topology from 0.239 at the input to 0.700 at block 8, against a baseline of 0.23 — while the structure model stays near it (0.400–0.481). This is also why the sequence model gets more chances to pick a best layer.

Three artefacts, each on its own large enough to manufacture a convergence result

Of the headline agreement between a sequence and a structure model, 69% of one measure is what it reports on scrambled data; roughly half of another is the training target both models share; and an untrained network scores 0.205 — a respectable-looking number — until that target is removed, whereupon it collapses to 0.038. Any one of the three would be enough to produce a convergence result where there is none.

The controls that catch them are cheap. A scrambled-residue null for every measure, so a score is read as its distance above chance, not as a bare number (Gröger et al., 2026, make the same argument for vision and language). Dimension matching before comparing models of different widths. A ceiling from the same model trained twice, so “high” has a reference. And, wherever two models are trained toward the same target, a residualisation of that target before agreement is measured — the target-side counterpart of the input-side deconfounding proposed by Cui et al. (2022). The last of these is the one this study adds, and it changed the answer.

What this does not show

References