Does the protein layer pick better drug targets?

The question

In breast cancer, does adding protein (and DNA) evidence to RNA nominate drug targets that are more often confirmed by held-out evidence than RNA alone — beyond what random genes matched for popularity achieve — and do the nominations replicate in an independent cohort?

And a second question: when an AI agent writes the rationale for a nominated target, how many of its claims survive a check against the held-out data, and does it prefer famous genes?

Expected answer, stated before any data: little or no list-level gain from the protein layer, because the genes that are druggable tend to be the ones whose mRNA and protein levels agree. A clean null is a measurement here: it tells a target-discovery team when proteomics is worth paying for. Either answer is reported.

Why this study

Study 1 of this series asked a related question of single-cell foundation models — do the cell states they find in breast tumours predict survival better than a linear baseline? — and read the answer against the same kind of ladder. Its result was a clean null.

Design

Three layers from the 77 TCGA-BRCA tumours that CPTAC profiled (Mertins 2016) feed six nomination arms. Each arm ranks one shared gene universe and returns its top 50 per PAM50 subtype (basal-like, HER2-enriched, luminal). The lists are scored against evidence no arm sees.

The arms

Per layer, a gene's evidence is a standardised mean difference between the subtype's tumours and the cohort's others (for DNA: copy-number difference with a cis-expression check, or mutation recurrence). A gene enters any list only if it is up in tumour against adjacent normal tissue; the same gate, from the same RNA data, applies to every arm.

Held-out truth

Dependency (primary). DepMap 24Q4 CRISPR gene effect in the 53 breast cell lines with a screen, each assigned to a subtype. The endpoint is the mean gene effect of a list in the matched lines (more negative means the list is enriched for genes those cells depend on); precision against a binary hit set is secondary. Pan-essential genes are excluded. Clinical precedence (secondary). Targets of drugs that reached any clinical phase for a breast indication, Open Targets 26.09, frozen.

The ladder

Every list is read against all five rungs. Arm differences carry a paired bootstrap interval over tumours; the smallest effect of interest is 0.05 in mean gene effect or 5 genes in 50.

Predictions, fixed before any data

Committed in DESIGN.md §8 on 2026-10-06. Any later change to the design is logged in its §12 with the commit and the reason.

Results

Most lists do not beat random genes of the same expression, popularity and protein detectability. Of the 6 arms, 0 clear the matched-random floor (5th percentile) in at least two of three subtypes. The strongest result is R-all in basal-like, better than 99.9% of matched random lists. In luminal tumours no arm beats the floor; the luminal truth rests on six cell lines. P1 therefore fails, and by the pre-registered rule P2–P6 below are reported, not interpreted.

Adding protein to RNA made the lists less dependency-enriched, not more. R+P minus R in mean gene effect: basal-like +0.159 [-0.017, +0.366]; HER2-enriched +0.151 [-0.094, +0.290]; luminal +0.003 [-0.058, +0.071] (positive = R+P worse). The interval excludes zero in 0 of 3 subtypes. P2 holds: the protein layer does not beat RNA by the smallest effect of interest in any subtype.

DNA was the layer that helped. R+D minus R: basal-like -0.090; HER2-enriched -0.202; luminal -0.257. Copy-number gains that track expression, and recurrent mutations, point at genes the matched cell lines depend on; subtype-specific protein abundance does not. P3 fails.

The integration method mattered little. MOFA+ minus R+P+D: basal-like -0.037 [-0.174, +0.257]; HER2-enriched +0.017 [-0.275, +0.287]; luminal -0.052 [-0.142, +0.085]. P4 fails.

RNA on many tumours against protein on few. Mean gene effect, R-all vs R+P: basal-like -0.772 vs -0.405; HER2-enriched -0.396 vs -0.219; luminal -0.117 vs -0.120. P6 holds.

Replication in the Krug 2020 cohort. Subtypes in which an arm's discovery and replication top-50 lists overlap more than the permutation null's 95th percentile: R 3 of 3; R+P 3 of 3; R+D 2 of 3; R+P+D 3 of 3; MOFA+ 1 of 3. The protein arm replicates no worse than RNA. P5 fails.

Each arm's top-50 against 1,000 random lists matched gene by gene on PubMed count, expression and protein detectability. A dot below the grey band beats the floor.

Differences between arms in the primary endpoint with 95% paired bootstrap intervals over tumours; the shaded band is the smallest effect of interest (±0.05).

Genes shared by the discovery and Krug 2020 top-50 lists, against the overlap when Krug's subtype labels are shuffled.

Candidate dossiers

The ten highest-ranked genes of the best-scoring arm per subtype, with every rung. "Dep." is the mean gene effect in subtype-matched lines (negative = dependency) and the number of those lines below −0.5; "clinic" is the highest clinical stage of a drug against the gene for a breast indication (4 = approved); "tumour cells" is the ratio of mean expression in malignant vs other cells of the 50,002-cell atlas; "HR" is the Cox hazard ratio per SD of expression with age and stage; "Krug" marks genes also in the replication cohort's list for the same arm.

basal-like — arm R-all

HER2-enriched — arm R+D

luminal — arm R+D

* common-essential gene (excluded from the hit sets). Full tables with every column: results/dossier/candidates.csv.

The agent audit

Agentic systems now write target rationales. Published evaluations ask whether the system found the right gene or whether a wet-lab experiment worked; none has scored the rationale itself, claim by claim, against held-out evidence while controlling for how famous each gene is.

1. Items. The top-10 genes per subtype from the best arm (30 candidates) and 30 decoys drawn from the same popularity, expression and detection strata that no arm nominated. The agent is not told which is which.
2. Agent. One named model, version and knowledge cutoff stated, closed-book and open-book (read access to this repository's frozen tables, nothing else). It returns a target score and a list of typed claims: over-expressed in the subtype, up in tumour vs normal, dependency in matched lines, clinical precedence, restricted in normal tissue.
3. Check. Every typed claim is verified automatically against the frozen tables and labelled supported, contradicted or unsupported.
4. Report. Faithfulness per claim type and condition; whether the agent's scores separate candidates from decoys; how strongly its scores track PubMed count.

Model: claude-sonnet-5-5, 30 candidates and 30 matched decoys per condition, roles hidden. Every typed claim was checked against the frozen tables.

Closed-book, the agent was more faithful than predicted and more fame-driven than it should be. 89.6% of its checkable claims held up (P7 predicted under 80%, so P7 fails); the weakest claim type was overexpressed in subtype (76.2%). Its target scores tracked PubMed count at ρ = 0.57 and separated real candidates from look-alike decoys at AUC 0.64 only, so P8 holds: from memory, the agent rates genes by how well known they are. Open-book, with the evidence row in front of it, faithfulness rose to 97.4%, decoy discrimination to AUC 0.79 and the popularity correlation fell to ρ = 0.47 (P8 fails in this condition). The practical reading: an agent that writes target rationales needs the evidence in context; from memory alone its ranking is largely a popularity ranking.

Full claim table: results/audit/claims.csv. How the agent was run, and what that changes from §9 of the design, is DESIGN §12 entry 9.

From raw reads to a dependency

The tumour mutation calls above arrive pre-processed. To show the step from sequencer output to variants, the SEQC2 reference pair HCC1395 / HCC1395BL — a triple-negative breast-cancer line and its matched normal, with a published high-confidence somatic truth set — is aligned and called from raw exome reads (BWA-MEM2, GATK Mutect2) and the calls are scored against that truth set. HCC1395 is also a CRISPR-screened DepMap line, so its called mutations are traced into its own dependencies and into the basal-like candidate list.

PASS Mutect2 calls against the SEQC2 v1.2.1 high-confidence somatic calls, both inside the exome target ∩ high-confidence regions (80.7 Mb). Indel counts are small and indicative only. Coverage on target — HCC1395: mean 50×, 75% of the target at ≥ 20×; HCC1395BL: mean 65×, 85% of the target at ≥ 20×. Alignment of both samples took 0.7 h on 8 threads. Omitted and stated: base-quality recalibration, the germline resource, the contamination estimate.

Mutated genes that HCC1395 depends on

Genes with a PASS somatic variant in the target and a CRISPR gene effect ≤ −0.5 in HCC1395 itself (DepMap ACH-000699): 62 of 3352 mutated genes. The strongest fifteen:

Data and licences

Every file is recorded with its URL, release and sha256 in results/MANIFEST.sha256 when stage data runs. No data from Sanger Project Score or Cell Model Passports are used (non-commercial terms). Open Targets' association score is never an input: it contains dependency and literature evidence that would make the truth circular.

Literature

The survey behind the design, with a strength tag on every entry, is in research/literature.md. The references the design rests on most:

1. Mertins P et al.. Proteogenomics connects somatic mutations to signalling in breast cancer. Nature 2016. doi
2. Krug K et al.. Proteogenomic landscape of breast cancer tumorigenesis and targeted therapy. Cell 2020. doi
3. Savage SR et al.. Pan-cancer proteogenomics expands the landscape of therapeutic targets. Cell 2024. doi
4. Lin HK, Dai J, Pusztai L. Breast-cancer targets from CRISPR dependency screens, by subtype. Breast Cancer Res Treat 2025. doi
5. Pacini C et al.. A comprehensive clinically informed map of dependencies in cancer cells and framework for target prioritisation. Cancer Cell 2024. doi
6. Rappoport N, Shamir R. Multi-omic and multi-view clustering algorithms: review and cancer benchmark. Nucleic Acids Res 2018. doi
7. Herrmann M et al.. Large-scale benchmark study of survival prediction methods using multi-omics data. Brief Bioinform 2021. doi
8. Gonçalves E et al.. Pan-cancer proteomic map of 949 human cell lines. Cancer Cell 2022. doi
9. Minikel EV et al.. Refining the impact of genetic evidence on clinical success. Nature 2024. doi
10. Haynes WA, Tomczak A, Khatri P. Gene annotation bias impedes biomedical research. Sci Rep 2018. doi
11. Chang L et al.. Systematic profiling of conditional pathway activation identifies context-dependent synthetic lethalities. Cancer Cell 2021.
12. Fang LT et al.. Establishing community reference samples, data and call sets for benchmarking cancer mutation detection (SEQC2). Nat Biotechnol 2021. doi

Limits, stated in advance

• Small cohorts. 77 discovery tumours with protein, 122 for replication; 53 breast cell lines, of which only 6–11 are luminal. The luminal arm may be uninformative.
• Weak, partly confounded truth. Dependency tracks expression (hence the matched floor); cell lines drift from tumours; ESR1- and ERBB2-type expression addictions do not reproduce in organoid screens and are flagged. Clinical precedence is small and popularity-biased.
• Asymmetric layers. CPTAC breast has no normal-tissue proteomics, so the tumour-vs-normal gate is RNA-based in every arm.
• Two integration methods only. Equal-weight rank stacking and MOFA+, neither tuned on the truth. Better integration methods remain untested here.
• Knowledge cutoff. The held-out data pre-date current language models' training data, so the agent audit leans on decoys and claim-level checks, with a time-split secondary check.