Blog
How does K Pro compare to Claude?
Same model underneath. Different harness on top. Ask the same scientific question twice and K Pro analyzes the same patients 90 times out of 100; Claude, handed the same processed data, 58.
What K Pro is
K Pro is an agentic AI platform built for biopharma R&D: target discovery, translational analysis, the daily work of computational biology. To earn a place in that work, where a wrong turn costs months and real money, a system has to be accurate, reproducible and traceable enough that a scientist can defend its output to a committee.
K Pro orchestrates a frontier language model. So does every serious agentic product, and the model is one anyone can rent. What differs is the harness: everything sitting between the model and the data. The orchestration that decides what happens in what order. The purpose-built tools that do the work. The domain expertise built into them. The model provides cognitive capacity, whereas the harness directs the reasoning and uses up-to-date data sources and computation to validate that direction.
Above the model sits something that is not rented — and that you can shape around your own work. Claude is a frontier model plus a general-purpose harness, built to be excellent at almost anything. K Pro is a comparable model plus a harness built by computational biologists, biostatisticians, ML researchers and clinical scientists for one domain, refined over a decade of collaborations with big-pharma partners, running on a governed data layer we built — and extensible to the analyses your programs actually run, rather than to a generic benchmark.
So the fair question from anyone evaluating us is: what does a domain harness actually buy you, if the model underneath is the same?
Three things, and then the evidence for each:
- Reliability, where reliability decides. Ask the same question twice and K Pro analyzes the same patients 90 times out of 100. Claude, handed the identical data, 58.
- A layer you can shape around your own work. The model is rented; the harness is not. It extends to your analyses, your conventions, your data — so the platform gets closer to your programs over time instead of staying generic. This is the part no frontier model can sell you.
- And you give up nothing to get them. On everything we did not tune for, K Pro is level with Claude: 56 to 58 on Anthropic's own BioMisteryBench benchmark.
What this means for pharma R&D and drug development
You have programs to run and decisions to defend, and the question that matters is not whether a system can find an answer once. It is whether the answer holds when a colleague re-runs it the following week.
When a frontier model improvises a cohort selection, it is effectively a confident, well-formatted analysis of a patient population nobody actually chose, and nothing on the page says so.
So while the industry competes on accuracy benchmarks, we built ours focusing on reliability. Ask the same scientific question twice — does the system analyze the same patients?
Handed the same processed public data, Claude did so in 58 of 100 repeat pairs. K Pro did in 90.
That is not luck. K Pro's analyses are not improvised from the prompt on each request; they run through tools we built and evaluated, so the same question takes the same route to the same patients. Here is how it was measured.
The benchmark
Ten use cases drawn from the published oncology literature, across eight TCGA cohorts. Each is an ordinary analysis a computational biologist runs routinely: split patients by a gene's expression and test survival, compare tumor against adjacent normal tissue, map mutation hotspots, cluster patients on a gene panel. We mostly chose mid-tier papers (full list in the appendix), because a benchmark built from landmark studies measures memorization rather than analysis.
Every question runs five times, in three conditions that mirror the three options on a buyer's desk:
- Claude, as it comes. The question alone; the agent needs to find TCGA by itself. This is how a coding agent actually gets used against public data.
- Claude, handed our processed public data. The same agent, pointed at K Pro's ingested, versioned TCGA package. The steel-man: it assumes the buyer builds the data engineering themselves.
- K Pro. The same model, the same data package, plus K Pro's harness — orchestration and purpose-built tools.
Conditions 2 and 3 are the ones that matter, because they differ in exactly one thing: the harness. Same model, same data, same sandbox, same questions, matched spend. 150 analyses in total.
Read the accuracy rows alone and you would conclude the three options are close. Six points separate the agent as shipped from the product, and building the data engineering yourself buys back most of that.
The top two rows are the ones that decide whether you can use the output. Governed data buys back accuracy. What it does not buy is the property a lab runs on: that the same question, asked twice, analyzes the same patients. That takes a harness, one that owns the decisions a question never specifies: which patients are in, how the analysis runs, how the result is reported.
One question, fifteen runs
The clearest way to show what that means is to stop aggregating and look at a single question in detail.
In TCGA stomach adenocarcinoma, classify patients as MUC16-mutant or MUC16 wild-type from somatic mutation calls. (1) Compute the tumor mutational burden of each patient and compare it between the two groups with a suitable test, shown as a violin or box plot. (2) Using the clinical N stage, is MUC16 mutation status associated with lymph-node metastasis (N0 vs N+)? Report the proportions and a test.
An ordinary request; a computational biologist would recognize it as a morning's work. But before any statistic can be computed, it forces four decisions the question itself does not settle:
- which patients count as having usable mutation data;
- what counts as "MUC16-mutant" — any coding variant, or non-synonymous only;
- which patients enter the N-stage comparison;
- which statistical test to run.
Three of those four are exactly what a harness owns. So the three conditions differ in one thing: who makes them.
All fifteen runs were judged correct. Every condition concluded that the mutant group has higher TMB, and that there is no significant nodal association. On this question accuracy is not in dispute and cannot explain anything below. The verdicts are identical. The analyses underneath them are three different worlds.
Claude, raw: the data supply decides. Four runs pulled the GDC masked MAF (an open-access somatic mutation file) via the API — 431 patients, mutant defined as bearing at least one somatic mutation, giving 166/265. Run 4 got a partial pull of 198 patients, less than half the cohort, applied a stricter non-synonymous-only definition, and landed at 76/122. Its TMB p-value is 0.0001 where the others sit near 4×10⁻²². That is a spread of eighteen orders of magnitude, driven by how much data the download happened to return. Nothing in run 4's answer flags that it analyzed half a cohort.
Claude, given our processed public data: the analytic convention decides. The data is now fixed, so the drift moves upstream into definitions. Two clusters: runs 1, 3 and 4 counted any coding somatic mutation in MUC16, giving 170 mutant; runs 2 and 5 counted non-synonymous only, giving 140. On top of that the wild-type count took five different values — 241, 242, 246, 271, 278 — as each run worked out its own join against the clinical table.
The consequence that matters is the clinical question. The nodal p-value moves from 0.134 to 0.49, a factor of 3.6, purely on which variant rule the run happened to pick. Here the true effect is comfortably non-significant, so nothing flips. On a borderline case, which way the study reads could turn on a decision nobody made deliberately and nobody wrote down.
K Pro: the harness decides. Cohort: the 416 STAD patients with whole-exome data. Mutant: at least one somatic MUC16 mutation, following the convention its tools apply. 140/276, five times out of five. TMB p = 4.4×10⁻¹⁷, five times. Nodal p = 0.151, five times. The only variation across five runs is print precision — 4.4, 4.42, 4.417. Even the sub-denominator for the N-stage comparison held.
You could ask which mutant definition is right — 140 or 170? Both are defensible, and that is the point. A choice that can go either way should be made once, by people who are accountable for it, and then applied consistently. That is what a harness does, and what an agent working from the prompt alone cannot.
Other notable examples
On the thyroid immune-correlation question, Claude, handed our processed public data, computed the same correlation over 572, 448, 572, 572 and 505 tumors — three denominators, 124 patients apart, none flagged as a choice.
On head-and-neck differential expression, K Pro reported between 2,064 and 2,067 up-regulated genes across five runs where Claude reported 1,994, 2,213, 2,215, 2,280 and 2,281. And in the raw condition, one run of that question silently lost four fifths of its cohort and returned a confident answer with 7,420 differentially expressed genes and a volcano plot. That run scored, too — a well-formatted analysis of the wrong cohort, with nothing on the page to reveal it. That is the failure mode that costs a pharma program months.
And on raw capability, we are level
Worth saying plainly, because it is the other half of the answer: choosing K Pro does not cost you analytical power.
BioMysteryBench is the bioinformatics benchmark Anthropic published. Each of its 90 problems hands a system a pile of anonymized biological data and asks it to deduce one hidden fact — which organ these cells came from, which gene was switched off — with every identifying clue stripped out. The answer has to come out of the data. It is hard: 17 of the 90 problems defeated every human expert on Anthropic's panel.
We ran it with K Pro against Claude, one attempt per question, in the same secure sandbox, with the same model under both. K Pro solved 56 of 90. Claude solved 58. A two-question gap on a test that size is noise, and we would say the same if it had fallen our way. On 80% of questions the two reach the identical outcome — right together or wrong together — and in three whole categories, covering 43 of the 90 questions, they score exactly the same.
K Pro works on real data, at real size. Nearly half the benchmark's problems come with files far too big to simply upload — some as large as 27 GB. Handling them means reading the data where it sits, under access control, without copying it anywhere less safe. That is plumbing rather than intelligence, and it is where most platforms stop. K Pro matched a frontier agent on it.
Single-shot accuracy is a property of the model, not the harness. When two systems fail the same questions in the same way, the capability being measured travels with the model — which means every vendor renting that model has it. Which is why we did not stop there.
Put the two results side by side and the arithmetic is simple. Level on the general capability anyone can rent. Ahead on the reliability nobody can rent. And tunable to the questions your programs actually run.
This benchmark is the floor, not the ceiling
Every number above came from a deliberately narrow setup: public molecular data, a model anyone can rent, general-purpose conventions, and no per-question tuning of any kind. That was the point — a comparison is only credible where the ground is shared, and this one is reproducible by anyone. It also means the benchmark measures K Pro's baseline, not its reach.
What sits beyond it:
- Skills. Pre-encoded domain workflows built with scientific experts, for the analyses that recur in real programs — target dossiers, translational readouts, the work that decides whether a program advances. We are building these out continuously, with the people who run those workflows for a living. Although Skills are part of K Pro's full harness, we turned them off for this benchmark: with Skills on we could have written one Skill per question and effectively pre-loaded the answers, which would have measured our ability to write Skills rather than the platform. A fair comparison would also have meant equipping Claude the same way.
- Capabilities this benchmark never touched. K Pro reads whole-slide H&E images through purpose-built pathology models, visualizes spatial biology, and runs competitive and landscape intelligence around the target a program is built on. These are not things a general-purpose coding agent does less well — they are things it does not do, because they need multimodal data, domain models and dedicated tools that no general-purpose harness ships with. A benchmark built on public molecular data cannot show any of it.
- Your conventions, your data. The harness extends to the analyses your team actually runs, against your own conventions and SOPs, applied to your proprietary data in a governed environment — with Owkin's proprietary models on top.
So, we match a frontier agent on its own ground (BioMysteryBench), and then keep going.
So: how does K Pro compare to Claude?
The model is the same. The harness is not, and that is the whole answer.
Where it matters, K Pro is stronger. Ask the same question twice and it analyzes the same patients 90 times out of 100 against 58, holds a single cohort on 8 of 10 questions against 3, and reproduces the published finding 92% of the time against 86% — with a route to the answer that can be inspected and repeated, because the decisions were made by people accountable for them rather than improvised on the spot.
And it is yours to shape. The harness extends to your analyses, your conventions and your data. We are building out expert-verified Skills with the scientists who run pharma's real workflows — target dossiers, translational readouts, the analyses that decide programs — so the questions that matter most to you become the ones K Pro is most reliable on. No rented model can offer that, because there is nothing there to shape.
On everything we did not tune for, K Pro is level with Claude — 56 to 58 on Anthropic's own benchmark. So the choice between them was never a choice about capability. It is a choice about whether the analysis you commission can be defended, repeated, and built on.
Pointed at your data, with your conventions encoded and Skills on, the gap is wider than a public benchmark can show.
We plan to release the full benchmark soon.
Appendix: how to read the benchmark numbers
Model. For this experiment we chose Claude Sonnet 4.6 and ran it under every condition, in the same secure sandbox, so that the only thing varying between conditions is the data and the harness.
Spend. We matched spend rather than capping it — $21.36, $20.38 and $20.91 for the three conditions tested, across fifty runs each. A larger budget, or a more powerful model, would likely lift every accuracy number here, K Pro's included. We think it is unlikely to close the reliability gap, because reliability is not a property a model buys with more reasoning. Nothing in the model is deciding which patients to select; that decision sits in the harness or it sits nowhere. What more spend does buy is a fair question, and one we want to explore in a separate article.
Harness configuration. The K Pro condition used K Pro's orchestration and tools. Pre-encoded Skills were not enabled in any run.
BioMysteryBench. One attempt per question, no retries. All 180 answers were re-graded in a single pass by one judge at temperature zero, blind to which system produced each. Anthropic's own published results use a different question set and average five attempts per question, so they are not comparable to these and we have not tried to line them up.
TCGA reproduction. Ten questions, five independent runs per question per condition; 150 analyses. Answers were graded by one rubric-based judge at temperature zero, blind to source. Reported values were extracted by a language model at temperature zero, blind to source; extraction was run twice and agreed with itself on every double-extracted value.
Pairwise figures treat each question's five runs as ten unordered pairs, one hundred per condition. Each question is only ever compared against its own five repeats, never across questions, because each defines its own denominator.
Studies selected for our benchmark:
Authors
