Single-cell foundation models · breast cancer · survival

Do single-cell foundation models discover cell states that improve survival prediction?

In this benchmark, no. Marker-gene signatures derived from zero-shot scGPT and Geneformer did not outperform signatures from a linear baseline (PCA), and neither model added reproducible prognostic value beyond age and stage.

50,002single cells · 152 donors

1,094TCGA-BRCA patients · 151 deaths

1,979METABRIC patients · 1,143 deaths

Same atlas, the same downstream pipeline for every representation, outcome-permutation controls and an independent cohort. Conclusions apply to the tested checkpoints and their marker-gene signatures.

Natalija Stepurko · October 2026 · natalija-stepurko.com

Can representations learned from millions of single cells find clinically useful tumour cell states that a linear method misses? I compared two single-cell foundation models, scGPT and Geneformer, with a principal-component baseline on the same atlas of 50,002 breast-cancer cells. Each representation was clustered into cell states, each state became a marker-gene signature, and every signature was tested against patient survival in TCGA-BRCA.

Neither foundation model showed an advantage over the baseline. In the planned analysis no representation's best survival association exceeded a selection-aware permutation test; the signatures did not reproducibly improve prediction beyond age and stage; and the lack of a foundation-model advantage held in an independent METABRIC cohort with every signature fixed in advance.

The models did differ in what they represented: their cell states were less dominated by single donors than the baseline's. The study separates that difference in representation from an improvement in clinical usefulness, which it did not find.

Scope. The study evaluates marker-gene signatures from two zero-shot checkpoints, not everything their embeddings encode.

All the code is public. Every number and figure on this page comes from the pipeline in this repository: atlas assembly, embeddings, the controls and the METABRIC replication.Natalija-Stepurko/single-cell-fm-probing More of my work. Other studies and what I am working on now.natalija-stepurko.com

The question

Single-cell foundation models such as scGPT (Cui et al. 2024) and Geneformer (Theodoris et al. 2023) are pretrained on tens of millions of cells, and a common claim is that they learn cell biology that a linear analysis of the same data misses. For drug discovery and patient stratification the test that matters is whether that biology separates patients by outcome. This study asks whether cell states defined by these two models in a tumour atlas, carried into patient cohorts as marker-gene signatures, predict breast-cancer survival better than states from a linear baseline.

How the benchmark works

From single cells to patient outcomes

Scroll sideways to see all of it.

The three representations share everything except the representation itself: the same 50,002 cells, the same Leiden clustering, marker ranking and gene universe, the same scoring of each state's markers in 1,094 TCGA-BRCA tumours, and the same survival statistics. Each representation's selected signature is the one furthest above its matched random-gene benchmark. Five controls stand between a signature and a claim:

control	the question it answers
Selection-aware permutation test	Could the best signature look predictive only because hundreds were tested? Outcomes are shuffled 10,000 times and the best of a representation's 201–369 signatures is kept each time.
Matched random-gene benchmark	Does a signature beat an arbitrary gene set of the same size and expression? In breast cancer most random gene sets predict outcome (Venet et al. 2011); 200 are drawn per signature.
PCA baseline	Does pretraining add anything to a simple linear representation of the same cells? Principal components of the highly variable genes (HVG-PCA, called PCA on this page) go through the identical pipeline.
Clinical references	Does a signature improve on what clinicians already have? Age and stage, the PAM50 intrinsic subtype (Parker et al. 2009) and the published 11-gene proliferation score (Nielsen et al. 2010).
METABRIC replication	Does a selected signature behave the same way in entirely different patients? Signatures and their directions are frozen before METABRIC outcomes are examined.

The technical report calls this set of controls the ladder and reads every result against all of its rungs.

Result 1 · Neither foundation model outperforms PCA

Survival prediction is measured with the concordance index (C-index): of two patients, how often the one with the higher score dies first, accounting for censoring. 0.5 is chance; higher is better discrimination (Harrell et al. 1982).

No representation's best signature beats a selection-aware permutation test. In the pre-specified analysis, where a higher score must predict worse survival, p = 0.20 (PCA), 0.098 (scGPT) and 0.37 (Geneformer). In a post hoc analysis that also counts protective signatures, PCA (0.025) and scGPT (0.041) clear the test and Geneformer (0.055, 2.0 Monte-Carlo standard errors above 0.05) does not.

Figure 1A Does any representation beat the selection-aware null?

None of the three representations passes the selection-aware test in the pre-specified analysis. For each representation, 10,000 shuffles of the survival outcomes show how well its best signature could do by chance when hundreds are tested; the observed best stays inside that null.

These p-values belong to each representation's best survival-performing signature. The selected signature in Figure 1B is chosen by a different rule (distance above its random-gene benchmark) and can be another signature; its own p-values against the same null are 0.67 (PCA), 0.098 (scGPT) and 0.59 (Geneformer).
Filled dots: p < 0.05. The post hoc panel also counts signatures whose higher score predicts longer survival; it is exploratory.

The foundation models trail the baseline. Above the matched random-gene benchmark, scGPT's selected signature trails PCA's by 0.021 in C-index (95% interval −0.078 to +0.012) and Geneformer's by 0.039 (−0.084 to −0.001); in the post hoc reading the differences are −0.010 and −0.013, with intervals spanning zero. scGPT's selected signature has the higher raw C-index (0.576 against 0.547), but random gene sets of the same size and expression score higher for it too (0.502 against 0.453), and the advantage disappears once that is subtracted.

Figure 1B Does a foundation model improve on PCA?

Neither foundation model shows a positive advantage over PCA. Each model's selected signature is compared with PCA's after subtracting what matched random gene sets achieve; negative values favour PCA.

Intervals are 95% patient-bootstrap intervals that re-select the signatures in every resample, so they include the cost of selection.

The PCA signature in this comparison is not an idealised biological reference. It went through the same selection rule and comes from a state confined to one dataset whose markers are testis and colon genes (see the selected states); it was selected because its random-gene benchmark is low.

Result 2 · No added value beyond age and stage

Age and stage alone reach a cross-validated C-index of 0.751 in TCGA-BRCA (1,072 patients, 142 deaths). Adding a selected signature from PCA, scGPT or Geneformer changes it by less than 0.002 in either direction (−0.0016 to +0.0008), and no signature improves it consistently across the five repeats. The published proliferation score, a fixed gene list with no selection, raises it by +0.008; once PAM50 subtype is in the model its change is −0.003. The selected cell-state signatures therefore show no incremental prognostic value in this evaluation.

Figure 2 Adding the cell-state signatures does not improve prediction beyond age and stage

No selected signature improves on age and stage. Each dot is the mean change over five repeats of cross-validation; the signature, and its direction, are re-selected inside every training fold, so the test patients' outcomes never influence the choice.

Whiskers show the range across the five repeats on the same patients, not a confidence interval.
Hazard ratios for these models are in the technical report (Figure 12); they are nominal, because the signatures were selected on these outcomes.

Result 3 · No advantage in an independent cohort

Signatures and their directions were fixed before METABRIC outcomes were examined. No signature was re-selected in the validation cohort.

In 1,979 METABRIC patients (1,143 deaths) (Curtis et al. 2012; Pereira et al. 2016), all eight foundation-model-minus-PCA differences are negative, for overall and disease-specific survival. The 95% interval lies entirely below zero for one of the four pre-specified comparisons and for all four post hoc ones, so the validation does not support a foundation-model advantage; it does not establish that every difference is negative.

Figure 3 The absence of a foundation-model advantage persists in an independent cohort

Points show each foundation model's advantage over PCA after correcting for matched random-gene performance; positive values would favour the foundation model. Filled: pre-specified; open: post hoc.

TCGA intervals re-select signatures in each bootstrap; METABRIC intervals are paired patient bootstraps with the signatures held fixed.

Some individual signatures do replicate, and that is consistent with the comparison above. Both PCA signatures replicate on both endpoints; Geneformer's pre-specified signature meets the replication criterion for disease-specific survival by 0.0015 in C-index and not for overall survival. The published proliferation score replicates most strongly (C-index 0.583 for overall and 0.637 for disease-specific survival, against random-gene 95th percentiles of 0.545 and 0.569).

What the models actually learned

If the foundation models do not improve survival prediction, what do their cell states capture? Two things stand out: they recover recognisable breast-cancer programmes, and their states are less dominated by individual donors.

Figure 4 What kinds of cell states did the models find?

The foundation models produce fewer donor-dominated states, and recover recognisable breast-cancer programmes. Panel A names the selected state of each representation in the two analyses; panel B counts every state kept for analysis, not only the selected ones.

A lower share of donor-dominated states is not by itself a better representation: malignant cells cluster by patient in single-cell data, so a donor-specific state can be genuine tumour biology as well as a batch effect.

Recognisable biology. Four of the six selected signatures are tumour-cell programmes whose bulk scores follow the PAM50 intrinsic subtypes, in TCGA and again in METABRIC: scGPT's pre-specified state is an ERBB2-containing luminal epithelial programme and Geneformer's a basal keratin programme; in the post hoc reading PCA and scGPT both recover a luminal, oestrogen-responsive programme. The other two are PCA's pre-specified state, confined to one dataset, and Geneformer's post hoc state, a profile of stressed or low-quality cells.

Less donor-specific states. 9% of scGPT's and 14% of Geneformer's states draw at least 80% of their cells from one donor, against 41% of PCA's. This did not give clearly stronger prognostic signatures: the best state shared across donors lies 0.074 (scGPT) and 0.035 (Geneformer) above its random-gene benchmark, against 0.075 for PCA; read in either direction the foundation models lead by 0.010 and 0.002, without an interval. The clusterings are stable under donor subsampling (mean adjusted Rand index 0.80–0.93).

Figure 5 places the selected states in each representation's two-dimensional UMAP of all 50,002 cells; Figure 6 shows every state each representation produced, at each of three clustering resolutions (123 for HVG-PCA, 67 for scGPT, 71 for Geneformer).

Figure 5 Where the selected states sit in each representation

Figure 6 Every cell state in each representation

Biology without incremental clinical value. A signature can follow a known subtype closely and still add nothing beyond age and stage. Among the six selected signatures, the subtype association ranges from ε² 0.03 to 0.43, while every change in C-index stays below 0.002. Only the published proliferation score, the most subtype-associated of all (ε² 0.61), adds a measurable amount.

Figure 7 Subtype signal and clinical utility

Tracking a breast-cancer subtype did not make a selected signature add prognostic value. Each point is one selected signature (or the published proliferation score): its association with PAM50 subtype against its change in cross-validated C-index over age and stage.

Six outcome-selected points do not establish a general relationship; the subtype measure uses the tumours with a PAM50 call and the C-index change uses the age-and-stage cohort.

What the results mean

The foundation models learned different cell-state representations, but the differences did not translate into better prognostic signatures. scGPT and Geneformer produced fewer donor-dominated states than PCA, and several of their selected states are recognisable breast-cancer programmes. Their marker-gene signatures nevertheless showed no evidence of outperforming PCA's, gave no reproducible improvement over age and stage, and showed no advantage in METABRIC.

This does not show that single-cell foundation models lack useful biological information. The checkpoints tested, scGPT whole-human and the 6-layer Geneformer V1, were used zero-shot; neither was pretrained on malignant cells, and cancer-adapted checkpoints such as CancerFoundation (Theus et al. 2024) were not tested. Turning a cell state into a short marker-gene signature scored in bulk tumours discards much of the representation. The result is specific to these checkpoints, this transfer procedure and breast-cancer survival, with 151 deaths in TCGA.

The broader lesson is methodological: a biologically plausible representation should not be assumed to add predictive value downstream without controlled baselines and independent validation. Representation structure, biological interpretability and clinical utility need to be measured separately.

What this study contributes

The study evaluates whether cell states found by single-cell foundation models carry survival information beyond a linear representation of the same cells. Its contribution is a controlled comparison: the downstream workflow is fixed across scGPT, Geneformer and PCA, the selection of promising signatures from hundreds of candidates is paid for, signatures are benchmarked against matched random gene sets, and the selected signatures are tested in an independent cohort. Earlier work established that single-cell states can be linked to patient outcomes (Luca et al. 2021; Wu et al. 2021) and that foundation models do not consistently beat simpler baselines (Kedzierska et al. 2025; Ahlmann-Eltze et al. 2025; Bendidi et al. 2024); this study connects the two by testing whether foundation-model cell states transfer into bulk-tumour prognosis, and by examining what the resulting states are.

Related work and how the findings compare

Simple methods match single-cell foundation models on cell-level tasks used zero-shot (Kedzierska et al. 2025) and on perturbation prediction (Ahlmann-Eltze et al. 2025; Bendidi et al. 2024), and foundation-model embeddings offer limited advantages for cancer patients' outcomes, whether used within single-cell cohorts (Roman et al. 2025) or computed from bulk tumour profiles (Liu W et al. 2026). Carrying single-cell states into bulk cohorts to study survival is established (Luca et al. 2021; Wu et al. 2021); here the foundation models decide which cells form a state.

The models recover familiar biology. Their states include an ERBB2-associated luminal programme, a basal keratin programme and a hormone-responsive luminal programme, and their scores follow the PAM50 intrinsic subtypes, the axes that carry most prognostic information in breast cancer (Parker et al. 2009; Wirapati et al. 2008).
That biology shows no evidence of adding survival information beyond the linear method. The linear method finds the same luminal programme, and no foundation-model signature predicted survival better than the baseline's, in TCGA or in METABRIC. This extends the cell-level and patient-level findings above (Kedzierska et al. 2025; Roman et al. 2025) to states the models define themselves.
An apparent advantage disappears against random genes. Random gene sets predict breast-cancer outcome (Venet et al. 2011), and here that background differed between the representations.
The signal that holds is protective. Signatures beat chance only when those predicting longer survival are counted (PCA and scGPT), and the states whose associations held across cohorts in Chen A et al. 2026, whose collection supplies 63% of this atlas's cells, were also protective.

The full literature review is in research/literature.md.

Research audit

The predictions were written down before any data were downloaded; post hoc analyses are labelled as such wherever they appear, and every departure from the original design is recorded with its reason. Inspect the design and its deviations, or open reproducibility and audit below.

Technical report

The full study, in four groups that open on demand: methods and controls, every number behind the summary, the secondary analyses, the deviations from the design, and how to reproduce it.

Terms used in the technical report: pick = a representation's selected signature; floor = its matched random-gene benchmark; family-wise null = the selection-aware permutation test; family maximum = the best survival-performing signature of a representation; nested ΔC = the change in cross-validated C-index with the signature re-selected inside every training fold; sensitivity analysis = the post hoc reading in which a signature may be risk or protective.

Study design and statistical controlsdata, representations, clustering, signatures, the four predictions, the full ladder of controls

Methods and controls

The three representations share everything except the representation itself: the cells, the clustering method, the marker ranking, the gene universe, the scoring and the survival statistics.

Three cohorts (OS: overall survival; PFI: progression-free interval; DSS: disease-specific survival)

The cohorts are independent collections; no atlas donor carries a TCGA barcode.

Assemble the atlas

CELLxGENE Census release 2025-11-08: every breast-carcinoma disease label, primary data only, droplet-based 3′ and 5′ chemistries. After quality control, 50,002 cells from 152 donors in 8 datasets were sampled in proportion to their 48 annotated cell types. No batch correction was applied; 63% of the cells come from one collection (Chen A et al. 2026). 577 cells (1.2%) duplicate another atlas cell exactly (identical raw counts, same donor), between that collection's Global Atlas and its compartment datasets; they were not removed (49,425 unique cells).

Represent every cell three ways

HVG-PCA: the 2,000 most variable genes, log-normalised, reduced to 50 principal components fitted on this atlas. scGPT (Cui et al. 2024): the whole-human checkpoint, pretrained on non-malignant human cells, cell embedding from its classification token. Geneformer (Theodoris et al. 2023): the 6-layer V1 model, pretrained on a corpus that excluded malignant cells, mean-pooled cell embedding. Both models are used zero-shot and run in bfloat16 on CPU.

Find cell states

Leiden clustering (Traag et al. 2019) of each representation's 15-nearest-neighbour graph at resolutions 0.3, 0.5, 1.0; states with at least 200 cells are kept: 123 HVG-PCA, 67 scGPT and 71 Geneformer states.

Turn states into signatures

The top 25, 50, 100 marker genes of each state (Wilcoxon test against all other cells), drawn only from the 16,297 genes also measured in the bulk cohort (1,526 of them highly variable): 369, 201 and 213 signatures.

Score the signatures in TCGA-BRCA

Each of 1,094 primary tumours (Cancer Genome Atlas Network 2012) gets one score per signature: the mean of the signature genes' expression, each z-scored across patients. The ladder uses Harrell's C (Harrell et al. 1982) of that score alone, without covariates. A Cox model with age and stage gives each pick's hazard ratio, reported separately (Figure 12).

Pick and read

Each representation's pick is the signature furthest above its own random-gene floor across all nine settings (three resolutions × three marker counts), chosen on TCGA outcomes. The family-wise null tests the family maximum, the largest C among a representation's signatures; read against the same null, the picks have p = 0.67, 0.098, 0.59 (pre-specified; HVG-PCA, scGPT, Geneformer) and 0.072, 0.11, 0.16 (either direction). A patient bootstrap that re-selects the pick in every resample gives the P2 margins. P1–P4 are then read off the ladder.

The ladder of controls

Every C-index sits on a ladder of controls, and findings are stated as distances between rungs.

The ladder (rung positions illustrative)

Scroll sideways to see all of it.

null	Survival outcomes permuted across patients 10,000 times. In each permutation the largest C-index among all of a representation's 201–369 signatures is kept, so the null includes the selection of a winner. The family-wise p of the observed maximum is read against it, with its Monte-Carlo standard error; a p within two standard errors of 0.05 is marked borderline. The design specified 500 permutations; the results keep both sets, which agree on every verdict.
floor	200 random gene sets per signature, matched for size and for mean-expression decile. In breast cancer most random gene sets are associated with outcome, largely through proliferation (Venet et al. 2011); the floor is what a gene list of that composition scores.
baseline	The HVG-PCA states, derived, scored and selected identically.
test	The scGPT and Geneformer states.
references	Age + stage: cross-validated C 0.751 (1,072 patients; 0.732 on the 821 with a PAM50 call). PAM50 subtype alone (Parker et al. 2009): 0.525 cross-validated, 0.598 in sample. PAM50 + age + stage: 0.723. The published 11-gene proliferation score of PAM50 (Nielsen et al. 2010): C 0.569 against a floor mean of 0.480, permutation p = 0.0085 (10,000 permutations). Cross-validation is 5 × 5-fold.

What the baseline controls for

One input, three views, one scoring set

Scroll sideways to see all of it.

All three representations are computed from the same expression matrix, and their signatures are scored on the same bulk genes. Prognostic signal present in all three can come from that shared input: cell-type composition, tumour purity or proliferation. The baseline measures what the shared input yields through a linear, unsupervised representation, and every claim about the foundation models is stated as the margin over it.

Predictions and how they came out

Four pass/fail predictions, P1–P4, read off the TCGA-BRCA overall-survival ladder.

prediction	how it was tested	result
P1Foundation-model signatures reach a C-index above the floor, each read against the family-wise permutation null.	The family maximum's family-wise p against 10,000 outcome permutations; passes if p < 0.05 and the maximising signature lies above its random-gene floor.	scGPT fail Geneformer fail
P2Foundation-model signatures exceed HVG-PCA signatures by a margin whose bootstrap interval excludes zero.	Margin of C above the random-gene floor over the HVG-PCA pick; 95% interval from 1,000 patient bootstraps that re-select the picks in each resample; passes if the lower bound is above zero.	scGPT fail Geneformer fail
P3At least one foundation-model state is prognostic within a PAM50 subtype.	Best within-subtype C against 10,000 within-subtype outcome permutations, every representation tested, the baseline included; passes if p < 0.05.	scGPT fail Geneformer fail
P4Foundation-model signatures that pass P2 are enriched for genes outside the HVG set.	If P2 passes: a hypergeometric test for HVG depletion in the pick against the scored-gene universe, and the pick's HVG fraction ranked among the baseline's signatures.	not applicable: P4 is tested only if P2 passes

The ladder in detail

Every rung for every representation in one figure (Figure 8; Figure 1A and Figure 1B in the summary are drawn from it), then the numbers, the either-direction reading, the progression-free interval and the ladder at each setting. Family-wise p-values are from 10,000 outcome permutations.

Figure 8 The ladder: each representation's pick against its floor, its null and three references

Pre-specified direction

The family maxima of all three representations stay inside their family-wise nulls in the pre-specified direction, and the margin-selected picks shown in Figure 8 do not clear the null in either reading (p = 0.67, 0.098 and 0.59 pre-specified; 0.072, 0.11 and 0.16 either direction). Geneformer's interval lies below zero, with its upper bound −0.001 near zero. scGPT's pick has the higher C-index (0.576 against 0.547) and also the higher floor (0.502 against 0.453); the baseline pick it is measured against is the single-dataset state described below and does not itself beat chance (p = 0.67).

The ladder at each of the nine settings (resolution × marker count)

representation	resolution	markers	signatures	best margin, risk	minus baseline	setting p, risk	best margin, either	minus baseline	setting p, either
HVG-PCA	0.3	25	31	+0.062		0.31	+0.087		0.014
HVG-PCA	0.3	50	31	+0.095		0.38	+0.095		0.006
HVG-PCA	0.3	100	31	+0.063		0.28	+0.081		0.020
HVG-PCA	0.5	25	42	+0.064		0.23	+0.090		0.038
HVG-PCA	0.5	50	42	+0.091		0.41	+0.096		0.010
HVG-PCA	0.5	100	42	+0.066		0.27	+0.091		0.022
HVG-PCA	1.0	25	50	+0.078		0.13	+0.095		0.042
HVG-PCA	1.0	50	50	+0.091		0.11	+0.091		0.006
HVG-PCA	1.0	100	50	+0.077		0.12	+0.101		0.018
scGPT	0.3	25	17	+0.016	−0.046	0.94	+0.066	−0.021	0.22
scGPT	0.3	50	17	+0.006	−0.089	1.00	+0.059	−0.036	0.14
scGPT	0.3	100	17	+0.032	−0.031	0.86	+0.056	−0.026	0.046
scGPT	0.5	25	18	+0.009	−0.056	0.97	+0.064	−0.026	0.23
scGPT	0.5	50	18	+0.006	−0.085	0.99	+0.079	−0.017	0.14
scGPT	0.5	100	18	+0.029	−0.037	0.88	+0.086	−0.005	0.044
scGPT	1.0	25	32	+0.067	−0.012	0.10	+0.082	−0.013	0.16
scGPT	1.0	50	32	+0.074	−0.017	0.076	+0.091	−0.001	0.014
scGPT	1.0	100	32	+0.073	−0.003	0.098	+0.090	−0.011	0.028
Geneformer	0.3	25	16	+0.044	−0.017	0.37	+0.073	−0.014	0.20
Geneformer	0.3	50	16	+0.037	−0.058	0.29	+0.057	−0.038	0.16
Geneformer	0.3	100	16	+0.056	−0.007	0.29	+0.056	−0.025	0.060
Geneformer	0.5	25	22	+0.045	−0.019	0.24	+0.070	−0.019	0.24
Geneformer	0.5	50	22	+0.039	−0.052	0.39	+0.069	−0.027	0.19
Geneformer	0.5	100	22	+0.052	−0.014	0.33	+0.059	−0.032	0.074
Geneformer	1.0	25	33	+0.037	−0.041	0.62	+0.088	−0.007	0.13
Geneformer	1.0	50	33	+0.042	−0.049	0.48	+0.083	−0.009	0.094
Geneformer	1.0	100	33	+0.055	−0.021	0.40	+0.081	−0.020	0.026

Best margin: the largest C above the random-gene floor among the setting's signatures. Setting p: the family-wise p of the setting's largest C against the null restricted to that setting (500 permutations). At every setting both foundation models' best margin is below the baseline's, in both readings; in the risk direction no setting clears its own null (smallest p = 0.076, scGPT at resolution 1.0 with 50 markers).

Either direction (post hoc sensitivity analysis)

The pre-specified analysis reads the C-index in one direction: a signature scores high only when a higher score means worse survival. Protective signatures are common and include the largest departures from 0.5 (C down to 0.400). The sensitivity analysis reads every signature in the direction it acts on the full cohort; its floor is read the same way, and the family-wise null takes the largest max(C, 1 − C) over the same 10,000 permutations. P3 is tested for every representation, the baseline included, with 10,000 within-subtype permutations; pick and direction were fixed on the full cohort, which contains these patients.

representation	pick	C	floor	pick's p	family max C	family-wise p	family null	best subtype	C within	p	P3
HVG-PCA	ER+/PR+ luminal programme · protective	0.591	0.490	0.072	0.600	0.025	clears	luminal A	0.702	0.016	passes
scGPT	luminal ER programme · protective	0.583	0.492	0.11	0.593	0.041	clears	luminal A	0.668	0.050	fails (borderline)
Geneformer	nuclear-RNA / stress · protective	0.578	0.490	0.16	0.590	0.055	does not clear (borderline)	luminal B	0.627	0.15	fails

The signatures that clear the null are the family maxima; for HVG-PCA and scGPT these are different states from the picks carried into added value, replication and Figure 9. Geneformer's family-wise p (0.055) lies 2.0 Monte-Carlo standard errors above 0.05: it does not clear, and the result is borderline. P2 still fails for both foundation models in this reading, and none of the pre-specified picks passes P3 (p = 0.38–0.40). Figure 9 shows the lead replicated signature as survival curves in TCGA, where it was selected: the HVG-PCA sensitivity pick, chosen by the rule in its caption.

Figure 9 TCGA (discovery) survival curves for the signature that replicated in METABRIC

Secondary endpoint: progression-free interval

The progression-free interval (PFI) has fewer events than OS in this cohort (145 against 151) and is analysed with the same statistics. The pattern is the same as for OS, with one difference: on PFI, Geneformer's either-direction family clears its null (p = 0.049, borderline), which it does not on OS.

representation	family-wise p, pre-specified	null (pre-specified)	family-wise p, either direction	null (either direction)	P2 margin, pre-specified (95% interval)
HVG-PCA	0.19	does not clear	0.025	clears	baseline
scGPT	0.21	does not clear	0.0054	clears	−0.007 (−0.043 to +0.006)
Geneformer	0.17	does not clear	0.049	clears (borderline)	−0.005 (−0.043 to +0.001)

Additional biological analysesthe selected states in detail, PAM50 subtype, donor mixing, stability

The selected states in detail

Each representation's pick is the state whose signature lies furthest above its own random-gene floor: in the risk direction for the pre-specified analysis, in either direction for the sensitivity analysis. Composition is from the 50,002-cell atlas; markers are ranked by a Wilcoxon test of the state against all other cells.

pick	programme, and the genes that name it	composition in the atlas	top-ranked markers (ribosomal genes omitted)	direction	TCGA C
HVG-PCA pre-specified · 0.3/c21_k50	single-dataset artefact CEACAM7, TKTL1, FAM9C, INSL3	284 cells 3 donors (83%), 1 dataset (100%) exhausted T cell 77% ribosomal genes: 0% of the signature	WDR76, TKTL1, CEACAM7, FAM9C, TMEM86B, IKZF4, KCNMA1, CORO2B, CBFA2T3	risk	0.547
scGPT pre-specified · 1.0/c11_k50	luminal epithelium (ERBB2+) ERBB2, EPCAM, KRT7, KRT19, SPDEF, TFAP2A	735 cells 41 donors (44%), 5 datasets (47%) malignant cell 95% ribosomal genes: 0% of the signature	MGST1, MUCL1, HEBP2, GGCT, EPCAM, CD24, CYB5A, RAD21, DBI	risk	0.576
Geneformer pre-specified · 0.3/c14_k100	basal keratin programme KRT5, KRT14, KRT17, KRT6A, KRT6B, TACSTD2, SFRP1	1,657 cells 37 donors (91%), 6 datasets (50%) malignant cell 99% ribosomal genes: 7% of the signature	KRT17, CSTB, TACSTD2, SLPI, KRT14, ANXA2, KRT6B, PERP, MT1X	risk	0.546
HVG-PCA sensitivity · 1.0/c17_k100	ER+/PR+ luminal programme GATA3, XBP1, PGR, CA12, STC2, ANKRD30A, AZGP1, TRPS1	444 cells 8 donors (98%), 4 datasets (53%) malignant cell 99% ribosomal genes: 22% of the signature	STC2, FASN, CLIC6, MUC1, MALAT1, SLC7A2, CA12, ANKRD30A, KRT19	protective	0.591
scGPT sensitivity · 1.0/c8_k50	luminal ER programme GATA3, XBP1, TFF3, AGR3, SCGB2A2, AZGP1, TRPS1	1,083 cells 47 donors (48%), 6 datasets (53%) malignant cell 96% ribosomal genes: 38% of the signature	KRT19, COX6C, XBP1, SCGB2A2, KRT18, KRT8, TFF3, HNRNPH1, S100A14	protective	0.583
Geneformer sensitivity · 1.0/c14_k25	nuclear-RNA / stress MALAT1, NEAT1, XIST, JUND, FOSB, PPP1R15A	854 cells 69 donors (27%), 7 datasets (74%) malignant cell 36% ribosomal genes: 4% of the signature	MALAT1, NEAT1, JUND, SRRM2, XIST, DDX5, DDX3X, SNHG5, ARGLU1	protective	0.578

Composition: donors and datasets with, in brackets, the share of the state's cells from the largest one, then the most frequent annotated cell type and its share. A state is flagged single-donor when one donor supplies at least 80% of its cells. ER, PR: oestrogen and progesterone receptor; HER2: the ERBB2 gene product.

Most picks are tumour-cell programmes that follow the intrinsic subtypes. In the risk direction, scGPT's pick is an ERBB2-containing luminal epithelial programme and Geneformer's a basal keratin programme. In the protective direction, HVG-PCA and scGPT both recover a luminal, oestrogen-responsive programme (GATA3, XBP1, AZGP1, TRPS1 in both lists). Their marker lists contain genes of the subtype axes that carry most prognostic information in breast cancer (Wirapati et al. 2008), and their bulk scores track PAM50 subtype (next section). Malignant cells cluster by patient in single-cell data (Tirosh et al. 2016): Geneformer's pre-specified pick (91% of its cells from one donor) and HVG-PCA's sensitivity pick (98%) are essentially one patient's tumour cells, and HVG-PCA's pre-specified pick (83%) is the single-dataset state below.

The baseline's pre-specified pick is a single-dataset artefact. Its cells are annotated as exhausted T cells, yet its markers are testis and colon genes (CEACAM7, TKTL1, FAM9C, INSL3), and none of the canonical T-cell markers (CD3D, CD3E, CD3G, CD2, CD4, CD8A, PTPRC) is among its 50 genes; it does include IKZF4, a regulatory-T-cell transcription factor, and the cell-cycle genes CDK2 and ERCC6L. It was picked because its random-gene floor is low (0.453), which makes its margin large although its C-index (0.547) is not. The pre-specified P2 margins are measured against it.

Geneformer's protective pick is a mixed-lineage state. Its markers are nuclear-retained transcripts and immediate-early genes, the profile of low-quality or dissociation-stressed cells.

The UMAP figures of these states are in the summary: Figure 5 and Figure 6.

Do the states track PAM50 subtype? (post hoc)

Outcome-free: each pick's bulk score compared across intrinsic subtypes, in TCGA with the RNA-seq PAM50 call (842 patients) and in METABRIC with its PAM50 and claudin-low call (1,974 patients, no outcomes read).

Figure 10 The picks' scores by PAM50 subtype, in TCGA and in METABRIC

Four of the six picks track subtype. scGPT's risk pick is highest in HER2-enriched tumours (one-vs-rest AUC 0.88) and Geneformer's in basal-like tumours (0.89); the luminal protective picks of HVG-PCA and scGPT are lowest in basal-like tumours (AUC 0.06 and 0.24). METABRIC tracks the same subtypes in the same direction (AUC 0.75, 0.90, 0.14 and 0.28). The baseline's single-dataset pick and Geneformer's stress state barely differ by subtype (ε² 0.03 and 0.04 in TCGA); the proliferation score, for comparison, has ε² 0.61. With PAM50 subtype, age and stage in the Cox model, the two foundation-model risk picks no longer point to risk (hazard ratio per standard deviation 0.90 and 0.91, nominal p 0.34 and 0.41), while the luminal protective picks keep hazard ratios below 1 (0.74 and 0.81); these hazard ratios are nominal, because every pick was selected on these outcomes.

Subtype association of the six picks and the proliferation reference

signature	pick	ε², TCGA	subtype tracked most (TCGA)	AUC, TCGA	ε², METABRIC	AUC, METABRIC (same subtype)	HR/SD given age, stage, PAM50 (nominal p)
HVG-PCA	pre-specified · single-dataset artefact	0.03	normal-like	0.65	0.14	0.43	1.14 (0.19)
scGPT	pre-specified · luminal epithelium (ERBB2+)	0.20	HER2-enriched	0.88	0.23	0.75	0.90 (0.34)
Geneformer	pre-specified · basal keratin programme	0.34	basal-like	0.89	0.22	0.90	0.91 (0.41)
HVG-PCA	sensitivity · ER+/PR+ luminal programme	0.43	basal-like	0.06	0.43	0.14	0.74 (0.013)
scGPT	sensitivity · luminal ER programme	0.15	basal-like	0.24	0.17	0.28	0.81 (0.028)
Geneformer	sensitivity · nuclear-RNA / stress	0.04	HER2-enriched	0.36	0.07	0.37	1.06 (0.48)
Proliferation score	published reference	0.61	luminal A	0.08	0.52	0.16	1.05 (0.73)

ε²: Kruskal–Wallis H / (n − 1), a rank-based effect size of subtype on the score. Subtype tracked most: the subtype whose one-vs-rest AUC is furthest from 0.5; below 0.5 means lower in that subtype. METABRIC AUC: for the subtype tracked in TCGA. HR/SD: Cox hazard ratio per standard deviation of the score with age, stage and PAM50 subtype as covariates, likelihood-ratio p.

Donor mixing (post hoc)

Every kept state at the three resolutions, each entering once through its 50-gene signature. States nest across resolutions, so the counts and the Fisher p are descriptive.

Figure 11 Donor mixing of the states, and their survival signal

Malignant cells cluster by patient in single-cell data (Tirosh et al. 2016), and the atlas was pooled without batch correction. 50 of 123 HVG-PCA states (41%) draw at least 80% of their cells from one donor, against 6 of 67 scGPT (9%) and 10 of 71 Geneformer states (14%); the median state draws 40%, 18% and 15% of its cells from its largest donor. The foundation-model embeddings therefore produced fewer donor-specific states than HVG-PCA.

This did not translate into clearly stronger prognostic signatures. Among multi-donor states (no donor above 50%), the best margin above the floor in the risk direction is 0.075 for HVG-PCA, 0.074 for scGPT and 0.035 for Geneformer. Read in either direction it is 0.081, 0.091 and 0.083: the foundation models lead by 0.010 and 0.002, single values selected on TCGA outcomes and without an interval. In the either-direction reading the margin rises with the share from the largest donor in all three representations (Spearman ρ +0.29, +0.46 and +0.35): the states furthest from their floors tend to be dominated by one patient.

Donor mixing per representation, with margins and correlations

representation	kept states	single-donor	multi-donor	median share from largest donor	median donors per state	Fisher p vs HVG-PCA	best multi-donor margin, risk	best multi-donor margin, either	Spearman ρ, margin vs share, risk (p)	ρ, either (p)
HVG-PCA	123	50 (41%)	69	40%	11		+0.075	+0.081	+0.17 (0.067)	+0.29 (0.0013)
scGPT	67	6 (9%)	60	18%	80	2.1 × 10⁻⁶	+0.074	+0.091	−0.30 (0.012)	+0.46 (9.2 × 10⁻⁵)
Geneformer	71	10 (14%)	59	15%	66	9.7 × 10⁻⁵	+0.035	+0.083	−0.20 (0.097)	+0.35 (0.0032)

Single-donor: largest donor ≥ 80% of the state's cells; multi-donor: largest donor < 50%. Margin: C above the matched-random floor of the state's 50-gene signature, TCGA overall survival. Fisher p: single-donor share against HVG-PCA's.

Are the states stable? (post hoc)

20 subsamples, each keeping 80% of the donors (122 of 152, the same draw for all representations); the neighbour graph and Leiden clustering are rebuilt on the kept cells with the settings of the main analysis.

The clusterings reproduce well: the mean adjusted Rand index between the subsample clustering and the full clustering restricted to the kept cells is 0.80–0.93 across representations and resolutions. The picks are less uniform. The best-matching subsample cluster shares on average 0.44 (scGPT's luminal ER pick) to 0.90 (the baseline's single-dataset pick) of a pick's cells (Jaccard index). The single-donor picks are reproduced almost exactly whenever their donor is kept, which is what one patient's cells clustering together predicts.

Stability per representation and resolution, and per pick

representation	resolution	clusters, full	clusters, subsample	adjusted Rand index, mean ± SD	minimum
HVG-PCA	0.3	33.9	31.6	0.85 ± 0.09	0.62
HVG-PCA	0.5	45.9	40.0	0.80 ± 0.04	0.70
HVG-PCA	1.0	57.9	53.6	0.82 ± 0.04	0.75
scGPT	0.3	17.8	17.6	0.93 ± 0.03	0.87
scGPT	0.5	20.6	21.8	0.89 ± 0.05	0.77
scGPT	1.0	34.6	33.8	0.83 ± 0.03	0.76
Geneformer	0.3	16.0	15.6	0.91 ± 0.05	0.79
Geneformer	0.5	22.9	20.6	0.85 ± 0.04	0.77
Geneformer	1.0	35.0	33.5	0.87 ± 0.03	0.82

representation	pick	largest donor	cell Jaccard, mean	marker Jaccard, mean	repeats with cell Jaccard ≥ 0.5	cell Jaccard when its largest donor is kept (repeats)
HVG-PCA	pre-specified · single-dataset artefact	83%	0.90	0.87	90%	1.00 (16)
scGPT	pre-specified · luminal epithelium (ERBB2+)	44%	0.63	0.51	65%	0.76 (15)
Geneformer	pre-specified · basal keratin programme	91%	0.83	0.82	85%	0.97 (17)
HVG-PCA	sensitivity · ER+/PR+ luminal programme	98%	0.67	0.72	75%	0.89 (15)
scGPT	sensitivity · luminal ER programme	48%	0.44	0.46	45%	0.53 (16)
Geneformer	sensitivity · nuclear-RNA / stress	27%	0.70	0.56	90%	0.77 (15)

Cell Jaccard: overlap of the pick's kept cells with the best-matching subsample cluster (0 when all its cells were dropped). Marker Jaccard: overlap of that cluster's top markers with the pick's signature, recomputed as in the main analysis.

Additional survival analysesCox models and hazard ratios, multivariable models, METABRIC in detail

Clinical information

Each pick is added to a Cox proportional-hazards model of age and stage (1,072 patients, 142 deaths) and to age, stage and PAM50 subtype (821 patients, 116 deaths); Figure 12 shows the result. The ladder's C-indices are of the score alone; this section asks whether the score improves on what a clinician already has.

Figure 12 Each pick added to a Cox model of age + stage

Age and stage alone reach a cross-validated C-index of 0.751 (1,072 patients); the PAM50 subtype call alone reaches 0.525 (821 patients). In this cohort PAM50 subtype does not raise the cross-validated C of age and stage either (0.732 against 0.733 on the 821 patients with a call), so the test has little room to show added value.

The hazard ratios in Figure 12 are nominal: every pick was chosen on the outcomes of these patients. The change in cross-validated C-index repeats the selection, and the choice of direction, inside every training fold, so it carries no selection. On that measure every pick's change in C spans zero across the five repeats. The luminal protective picks of HVG-PCA and scGPT keep nominal hazard ratios below 1 with PAM50 in the model (0.74 and 0.81 per standard deviation), but under nested selection they do not raise the cross-validated C-index. The published proliferation score (a fixed, published gene list, so free of selection) raises it by +0.008 over age and stage; once PAM50 is in the model its change is −0.003. Of the family maxima, only HVG-PCA's either-direction maximum raises it under nested selection (+0.007), and not once PAM50 is in the model.

All of a representation's states at once

A ridge-penalised Cox model with every state of a representation (50-gene signatures: 123 HVG-PCA, 67 scGPT, 71 Geneformer) plus age and stage does not improve on age and stage alone (Figure 13); differences of a few thousandths are within the spread across fold seeds. Gradient-boosted survival models (XGBoost, Cox objective, fixed hyperparameters) fall below age and stage alone with every representation's states (0.703–0.730), so they are not read as a comparison between representations.

Models within one section share their cross-validation folds; sections use different fold seeds, so the same model can differ by up to 0.009 between sections (age + stage 0.750–0.752; PAM50 + age + stage 0.723 and 0.732).

Figure 13 Multivariable models with and without the states

METABRIC in detail

The signatures, their directions and the replication criterion were fixed before METABRIC outcomes were examined.

METABRIC (Curtis et al. 2012; Pereira et al. 2016) is an expression-microarray cohort with long follow-up: 1,979 patients, 1,143 deaths (OS). Disease-specific survival (DSS) is the secondary endpoint; it censors deaths from other causes, leaving 646 events. Twelve signatures were frozen with their genes and TCGA directions: the six picks, the family maxima and the proliferation reference. A signature replicates on an endpoint if its C, read in the TCGA direction, exceeds the 95th percentile of its METABRIC floor and its age-adjusted hazard ratio, stratified by METABRIC cohort, points in the TCGA direction with p < 0.05.

Figure 14 Frozen signatures in TCGA and in METABRIC

Numbers behind Figure 14 for the six picks and the reference

Overall survival

representation	pick	genes measured	C	floor p95	HR/SD (p)	replicates
HVG-PCA	pre-specified · single-dataset artefact	98%	0.572	0.551	1.14 (1.6 × 10⁻⁵)	yes
scGPT	pre-specified · luminal epithelium (ERBB2+)	98%	0.543	0.543	1.19 (2.8 × 10⁻⁷)	no (borderline)
Geneformer	pre-specified · basal keratin programme	98%	0.535	0.546	1.17 (6.0 × 10⁻⁷)	no
HVG-PCA	sensitivity · ER+/PR+ luminal programme	98%	0.555	0.527	1.10 (0.0013)	yes
scGPT	sensitivity · luminal ER programme	98%	0.529	0.521	1.01 (0.70)	no
Geneformer	sensitivity · nuclear-RNA / stress	88%	0.507	0.533	1.00 (0.97)	no
Proliferation score	published reference	100%	0.583	0.545	1.25 (2.9 × 10⁻¹³)	yes

Disease-specific survival, and overall survival within the TCGA lead subtype

representation	pick	C	floor p95	HR/SD (p)	replicates	within subtype, OS (no floor)
HVG-PCA	pre-specified · single-dataset artefact	0.592	0.569	1.26 (6.3 × 10⁻⁹)	yes
scGPT	pre-specified · luminal epithelium (ERBB2+)	0.560	0.561	1.26 (1.3 × 10⁻⁷)	no (borderline)
Geneformer	pre-specified · basal keratin programme	0.573	0.572	1.23 (5.3 × 10⁻⁷)	yes (borderline)
HVG-PCA	sensitivity · ER+/PR+ luminal programme	0.594	0.546	1.20 (9.2 × 10⁻⁷)	yes	luminal A: C 0.575, p 0.001
scGPT	sensitivity · luminal ER programme	0.544	0.545	1.06 (0.15)	no (borderline)	luminal A: C 0.562, p 0.001
Geneformer	sensitivity · nuclear-RNA / stress	0.532	0.558	1.07 (0.097)	no	luminal B: C 0.490, p 0.71
Proliferation score	published reference	0.637	0.569	1.44 (7.6 × 10⁻²¹)	yes

C is read in the direction fixed in TCGA. Floor p95: 95th percentile of 200 matched random gene sets in METABRIC. HR/SD: age-adjusted hazard ratio per standard deviation of the score in that direction, stratified by METABRIC cohort. Borderline: C within two approximate Monte-Carlo standard errors of the floor p95 (normal approximation from the floor mean and p95). Within-subtype p: 1,000 outcome permutations; 0.001 is the smallest value attainable.

The replication shows no foundation-model advantage (Figure 3). The published proliferation score replicates most strongly (OS C = 0.583 against a floor 95th percentile of 0.545; DSS C = 0.637). Both HVG-PCA picks replicate. Of the foundation-model picks only Geneformer's risk pick meets the criterion, on DSS and by +0.0015, within Monte-Carlo error of the floor p95. Within luminal A tumours scGPT's protective pick has C = 0.562 with outcome-permutation p = 0.001; no random-gene floor was computed within subtypes, so this is a result, not a pre-specified replication. Geneformer's protective pick does not replicate.

Random gene sets predict survival in METABRIC. With 1,143 deaths, the matched-random floors have 95th percentiles of 0.52–0.57. Some signatures have hazard-ratio p-values below 10⁻⁵ and still do not beat random genes: Geneformer's risk pick (OS), scGPT's risk pick (OS and DSS). This is the effect described by Venet et al. 2011.

The baseline's artefact pick replicates on both endpoints. Why it replicates was not examined; its 50 genes include the cell-cycle genes CDK2 and ERCC6L.

Reproducibility and auditdeviations from the design, reproduction tiers, limits, references

How the analysis was checked: deviations from the design

The code was audited against the design document. Where the reported analysis differs from the design, or goes beyond it, the table gives the design, what is reported and why; post hoc analyses are labelled as such wherever they appear.

Deviations from the design, and analyses added after it (17 items)

item	design	what is reported, and why
Clinical reference	The PAM50 subtype call and a published prognostic signature.	Age + stage, PAM50 subtype, and both together, each cross-validated on one set of patients; the published 11-gene proliferation score with its own random-gene floor and permutation p. A single in-sample model of PAM50 + age + stage on fewer patients would not be comparable with the single-score C-indices.
P1: what the family-wise null tests	A family-wise permutation null for the best signature's C-index.	The family maximum with its family-wise p, the statistic the null is built for; the margin-selected pick's p against the same null is reported beside it.
P1: number of permutations	500 outcome permutations.	10,000 permutations, for Monte-Carlo precision near 0.05; the 500-permutation values are kept and agree on every verdict.
P2: margin over the baseline	Test against baseline, both read as distance above the floor; the bootstrap interval must exclude zero.	Above-floor margin with 1,000 patient bootstraps that re-select each pick in every resample, so the interval includes the selection over hundreds of signatures; passes if the lower 2.5% bound is above zero.
P3: prognosis within subtype	At least one foundation-model state is prognostic within a PAM50 subtype.	Best within-subtype C against 10,000 within-subtype outcome permutations, for every representation, the baseline included, so that a pass can be compared with what the baseline reaches.
Monte-Carlo error	Decisions at α = 0.05.	Every permutation p is given with its Monte-Carlo standard error, and a p within two standard errors of 0.05 is marked borderline.
P4: genes outside the HVG set	Foundation-model signatures that pass P2 are enriched for non-HVG genes.	Tested only if P2 passes, with a hypergeometric test against the scored-gene universe; P2 fails, so P4 is not applicable.
Ladder at each setting	Resolution and marker count are swept, and the ladder is reported at each setting.	One pick per representation across all nine settings, paid for by a family-wise null over the same nine; the ladder at each setting, with its own null, is in the table under the ladder in detail.
Adjustment for age and stage	Cox models adjusted for age and stage in the primary model.	The ladder C-index is of the score alone; adjustment enters through the added-value analysis: hazard ratios given age and stage, and the change in cross-validated C with the selection redone inside every fold.
Bootstrap intervals in the ladder	Patient bootstrap intervals for the ladder.	The whiskers in Figure 8 hold the pick fixed and are conditional on its selection; the selection-aware interval is the one used for P2.
Penalised reference models	PAM50 subtype as a reference rung.	The cross-validated C is the figure compared; the in-sample C of the penalised and unpenalised fits are given because they differ for PAM50 alone.
Secondary endpoint	Progression-free interval reported beside overall survival.	PFI analysed with the same statistics.
Direction of effect	C-index read in the risk direction only.	Added post hoc: every signature also read in the direction it acts, labelled as a sensitivity analysis.
Clustering stability	A separate diagnostic.	Added post hoc: 20 donor subsamplings (80% of donors), adjusted Rand index per clustering and overlap per pick.
Subtype association and donor mixing	Not in the design.	Added post hoc and outcome-free: each pick's score by PAM50 subtype in TCGA and METABRIC, and the donor composition of every state against its margin.
Multivariable models	Not in the design.	Added: ridge and gradient-boosted Cox models of all of a representation's states with age and stage, the penalty applied to the state features only.
METABRIC replication	Validation of any shortlisted state in an independent cohort.	All six picks, the family maxima and the proliferation reference, with signatures, directions and the replication criterion fixed before METABRIC outcomes were examined.

Reproduction

The pipeline is a Python package, scfm, whose stages are named commands. Each prints its plan with --dry-run before it touches data.

Stages, commands and reproduction tiers

stage	does
data	assemble the single-cell atlas and the bulk cohort; harmonise genes
embed	one embedding per cell for HVG-PCA, scGPT, Geneformer
states	Leiden clusters per representation → marker-gene signatures
stability	clusters and picks under donor subsampling (needs the ladder; not part of 'all')
translate	score signatures in bulk; Cox; permutation null; matched-random floor; references
ladder	assemble the ladder; patient bootstraps; test P1-P4
stratify	cross-validated multivariable stratification: states vs age + stage
report	figures
replicate	frozen signatures in METABRIC (run on its own; not part of 'all')

uv run scfm list                          # the stages, in order
uv run scfm run all --dry-run             # every stage's plan, nothing is run
uv run scfm run translate -- --endpoint PFI
make reproduce && make verify             # tier A, then compare checksums of the outputs

Three reproduction tiers

A, from the distributed results (about 40 min): rebuilds translate, ladder, stratify and report for OS and PFI from the distributed signatures. B, from the per-cell embeddings (not distributed): reruns states onward from the distributed cell list; the donor-subsampling stability stage adds about 25 min on 6 cores. C, from scratch (about 2 h 45 min on CPU, of which embedding is 1 h 44 min). Times are on a shared CPU machine. Tier A does not rebuild the atlas UMAP (Figure 5), which needs per-cell tables that are not distributed.

Tests and continuous integration

pytest covers the statistics, the stages' helpers and the Geneformer tokeniser, which is checked against the official tokeniser on a fixture; continuous integration runs the tests on every pull request. A small run executes every stage except replicate, both foundation models included, on 2,000 cells. Downloads are checksum-verified before use, and the atlas release and model versions are pinned.

Limits

Neither model saw malignant cells in pretraining. The scGPT whole-human checkpoint was pretrained on non-diseased cells and Geneformer's corpus excluded malignant cells, while most of the picks here are malignant-cell states. Cancer-adapted checkpoints, such as CancerFoundation (Theus et al. 2024) or the cancer-tuned Geneformer, were not tested.
Light checkpoints, zero-shot. scGPT whole-human and the 6-layer Geneformer V1 model, without fine-tuning, run on CPU. A negative result is a result about these checkpoints.
A bulk score is a lossy projection. The mean z-score of a state's markers mixes the state's abundance with tumour purity, proliferation and subtype; information in the embedding that its marker genes do not carry is not tested.
One atlas, no batch correction. 8 datasets, 3′ and 5′ chemistries, pooled without integration; one baseline pick is confined to a single dataset.
Single-donor states. Three of the six picks, and 50, 6 and 10 of the HVG-PCA, scGPT and Geneformer states, draw at least 80% of their cells from one donor.
Duplicate cells. 577 cells (1.2%) duplicate other cells across Chen et al.'s Global Atlas and its compartment datasets; they were not removed.
Power. 151 deaths among 1,094 TCGA patients, with short follow-up; the TCGA Pan-Cancer Clinical Data Resource flags breast-cancer OS as needing longer follow-up (Liu J et al. 2018).
Selection in one cohort. The picks were chosen on TCGA outcomes. The family-wise null, the selection-aware bootstrap and the nested cross-validation pay for that choice within TCGA; METABRIC is the only out-of-sample test.
One indication. Breast cancer was chosen for cohort size and established references; nothing here extends to other tumours until it is run there.

References

The literature note in the repository covers the field in more detail.

Ahlmann-Eltze C, Huber W, Anders S. Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines. Nature Methods 22:1657–1661 (2025). doi:10.1038/s41592-025-02772-6
Bendidi I, Whitfield S, Kenyon-Dean K, Ben Yedder H, El Mesbahi Y, Noutahi E, Denton AK. Benchmarking transcriptomics foundation models for perturbation analysis: one PCA still rules them all. arXiv:2410.13956 (2024). arXiv:2410.13956
Cancer Genome Atlas Network. Comprehensive molecular portraits of human breast tumours. Nature 490:61–70 (2012). doi:10.1038/nature11412
Chen A, Kroehling L, Ennis CS, Denis GV, Monti S. A highly resolved integrated single-cell atlas of human breast cancers. NAR Genomics and Bioinformatics 8:lqaf217 (2026). doi:10.1093/nargab/lqaf217
Cui H, Wang C, Maan H, Pang K, Luo F, Duan N, Wang B. scGPT: toward building a foundation model for single-cell multi-omics using generative AI. Nature Methods 21:1470–1480 (2024). doi:10.1038/s41592-024-02201-0
Curtis C, Shah SP, Chin SF, et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature 486:346–352 (2012). doi:10.1038/nature10983
Davidson-Pilon C. lifelines: survival analysis in Python. Journal of Open Source Software 4:1317 (2019). doi:10.21105/joss.01317
Goldman MJ, Craft B, Hastie M, et al. Visualizing and interpreting cancer genomics data via the Xena platform. Nature Biotechnology 38:675–678 (2020). doi:10.1038/s41587-020-0546-8
Harrell FE, Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests. JAMA 247:2543–2546 (1982). doi:10.1001/jama.1982.03320430047030
Kedzierska KZ, Crawford L, Amini AP, Lu AX. Zero-shot evaluation reveals limitations of single-cell foundation models. Genome Biology 26:101 (2025). doi:10.1186/s13059-025-03574-x
Liu J, Lichtenberg T, Hoadley KA, et al. An integrated TCGA pan-cancer clinical data resource to drive high-quality survival outcome analytics. Cell 173:400–416.e11 (2018). doi:10.1016/j.cell.2018.02.052
Liu W, Wang Q, Long L, Wang W. Leveraging single-cell foundation models for accurate survival outcome prediction. Bioinformatics Advances 6:vbag076 (2026). doi:10.1093/bioadv/vbag076
Luca BA, Steen CB, Matusiak M, et al. Atlas of clinically distinct cell states and ecosystems across human solid tumors. Cell 184:5482–5496.e28 (2021). doi:10.1016/j.cell.2021.09.014
Nielsen TO, Parker JS, Leung S, et al. A comparison of PAM50 intrinsic subtyping with immunohistochemistry and clinical prognostic factors in tamoxifen-treated estrogen receptor-positive breast cancer. Clinical Cancer Research 16:5222–5232 (2010). doi:10.1158/1078-0432.CCR-10-1282
Parker JS, Mullins M, Cheang MCU, et al. Supervised risk predictor of breast cancer based on intrinsic subtypes. Journal of Clinical Oncology 27:1160–1167 (2009). doi:10.1200/JCO.2008.18.1370
Pereira B, Chin SF, Rueda OM, et al. The somatic mutation profiles of 2,433 breast cancers refine their genomic and transcriptomic landscapes. Nature Communications 7:11479 (2016). doi:10.1038/ncomms11479
Roman A, Johri S, Conci R, Van Allen EM, Elmarakeby H. Empirical evaluation of single-cell foundation models for predicting cancer outcomes. bioRxiv (2025; v2 2026). doi:10.1101/2025.10.31.685892
Sun D, Guan X, Moran AE, et al. Identifying phenotype-associated subpopulations by integrating bulk and single-cell sequencing data. Nature Biotechnology 40:527–538 (2022). doi:10.1038/s41587-021-01091-3
Theodoris CV, Xiao L, Chopra A, et al. Transfer learning enables predictions in network biology. Nature 618:616–624 (2023). doi:10.1038/s41586-023-06139-9
Theus A, Barkmann F, Wissel D, Boeva V. CancerFoundation: a single-cell RNA sequencing foundation model to decipher drug resistance in cancer. bioRxiv (2024). doi:10.1101/2024.11.01.621087
Tirosh I, Izar B, Prakadan SM, et al. Dissecting the multicellular ecosystem of metastatic melanoma by single-cell RNA-seq. Science 352:189–196 (2016). doi:10.1126/science.aad0501
Traag VA, Waltman L, van Eck NJ. From Louvain to Leiden: guaranteeing well-connected communities. Scientific Reports 9:5233 (2019). doi:10.1038/s41598-019-41695-z
Venet D, Dumont JE, Detours V. Most random gene expression signatures are significantly associated with breast cancer outcome. PLoS Computational Biology 7:e1002240 (2011). doi:10.1371/journal.pcbi.1002240
Wirapati P, Sotiriou C, Kunkel S, et al. Meta-analysis of gene expression profiles in breast cancer: toward a unified understanding of breast cancer subtyping and prognosis signatures. Breast Cancer Research 10:R65 (2008). doi:10.1186/bcr2124
Wolf FA, Angerer P, Theis FJ. SCANPY: large-scale single-cell gene expression data analysis. Genome Biology 19:15 (2018). doi:10.1186/s13059-017-1382-0
Wu SZ, Al-Eryani G, Roden DL, et al. A single-cell and spatially resolved atlas of human breast cancers. Nature Genetics 53:1334–1347 (2021). doi:10.1038/s41588-021-00911-1