A Practical Map of Evaluation for Agentic Systems

At Owkin, we have spent the last decade helping pharma identify therapeutic targets and design better clinical trials by combining the Owkin data network with specialized multimodal AI models. K Pro builds on this foundation: an agentic AI system designed to help scientists reason across biomedical evidence. This post shares what we learned building a multi-layer evaluation harness to assess K Pro across both structured biomedical workflows and open-ended research tasks, from trial optimization to discovering and prioritizing new candidate targets, biomarkers, and patient populations.

Why classical ML evaluation breaks on AI agents

Most machine learning evaluation follows the same pattern: give the model an input, compare its output to a ground truth, and aggregate a score. It works well when the model produces a single, deterministic output from a fixed input. Agentic systems break this assumption: a single prompt can invoke several tool calls, intermediate decisions, and multiple outcomes across runs. They can also fail in ways classical evaluation will not catch: a confident, well-formatted answer can be silently fabricated if the agent skips a required step. This calls for evaluation approaches that go beyond one-shot output matching.

An agentic system is typically composed of an LLM, i.e the "brain", that is connected to a set of tools, which constitute the "body." To answer a question, the system iterates over planning, calling tools and reading results before providing an answer. The path from a question to a final answer can span numerous steps all of which are prone to variation. The flexibility and reasoning capabilities that make these AI systems so powerful are also the source of two main challenges for evaluation: 

• Non-determinism. Running the same query twice may lead to two different tool-call sequences and therefore two different answers, both potentially correct. A single-run accuracy score cannot capture this; we need to measure the distribution of outcomes by running the question multiple times. The most useful way to reason about that distribution is what we call the agent’s unreliability budget: the gap between what it can do at least once and what it can do every time. Two metrics make this concrete. pass@k asks whether the system answered correctly at least once in k attempts, a measure of capability. pass^k asks whether the system answered correctly on every one of k attempts, a measure of reliability. A K Pro question that succeeds 9 times out of 10 is acceptable for hypothesis generation, where the scientist will review the output anyway. It is not acceptable for a clinical decision-support workflow, where the same gap means a one-in-ten failure rate at the point of care. The unreliability budget you can tolerate is set by the stakes of the task, not the model. 

• Trajectory variability. A correct final answer produced via the wrong sequence of tool calls is not a success. Sometimes unnecessary tools can be called to answer the query, bloating the cost and latency of the AI system.
  For example, the right gene query runs against the wrong database, or the right plot is produced from incomplete data. Evaluating only the final response is like reviewing code by running it once and checking if it did not throw an error. Like in interpretable machine learning where tools computing feature importance, like SHAP values, were used to understand how the output predictions were made, monitoring, evaluating and optimizing the trajectory of an AI system is key to making it reliable as well as cost and time-efficient. 

To take these challenges into account, we need a complete, layered evaluation approach. At Owkin, we have built a three-layered architecture for AI system evaluation, comprising tool evaluation, trajectory evaluation, and end-to-end evaluation. Here is an overview of how we built the evaluation harness of K Pro.

‍

Layer 1: Tool evaluation: the unit test of agentic systems

The first evaluation layer assesses each tool call as an independent unit. It answers the questions: 

1. Did the agent call the right tool? 
2. Did it pass the right parameters? 

Calling the right tool is relatively binary: e.g. either the agent invoked the clinical trials search or it did not. The second question is harder than it looks: passing the right parameters means the agent correctly understood the user's intent and translated it into a precise, structured query. For example, when a user asks about a cancer drug in a specific patient population, the agent must not only identify the right tool to use, but also pass as search parameters the right disease and the right treatment phase. Every time a developer changes something in K, there's a risk that the AI agent silently gets worse at performing a task that it was easily capable of doing before the update was made. This evaluation layer is the automated safety net that spots these incidents. It periodically sends hundreds of representative questions to the AI system, checks the answers against expected tool calls and parameters, and enforces minimum passing grades for each step of the process. These daily automatic checks turn what used to be weeks of manual testing into a continuous feedback loop while striking the optimal balance between minimizing cost and maximizing the speed of issue identification throughout the development lifecycle.

‍

Layer 2: Trajectory evaluation: right answer, right path

A tool that works correctly but in isolation can still be used incorrectly by the AI orchestrator. Whereas the first evaluation layer tests each tool call separately, the second layer asks a broader question: when the AI has access to all tools at once, does it follow the right sequence of steps to reach a correct answer?

To address this, we define, for each question in our test set, an expected trajectory which consists of an ordered series of tool calls the agent should make. For instance, 1) retrieve the relevant clinical trials data, 2) filter it, 3) generate a plot. The second evaluation layer checks whether the agent followed this path. Some trajectories require strict tool call order because each step depends on the previous one; others are more flexible and just require all the right steps to happen, in any order. The right choice depends on whether the logic is causal.

Beyond ordering, this layer also enforces tool call “presence” and “absence” checks. Presence checks whether a required tool was actually called. Absence checks whether an unneeded tool was indeed not called and this is equally important , because an agent that reaches for the wrong tool doesn't just fail to answer but may also produce a confident-looking answer that is entirely fabricated. Consider, for instance, asking K Pro to summarize the latest literature on a drug target: if it silently bypasses the literature search and answers purely from memory, the output looks fluent and well-formatted, but entirely misses the user's expectation. This failure mode is harder to detect than a system crash and is significantly more dangerous in a scientific context where users might act on the information they received.

The result of this evaluation layer is a match rate, in other words, the fraction of expected steps that were satisfied. Internal thresholds and quality requirements are set on this evaluation layer, which drive our releases and internal development. 

‍

Layer 3: End-to-end evaluation and LLM-as-judge

The first two layers verify the mechanism. Layer 3 asks the question that actually matters: did the agent satisfy the user's intent?

For questions with a single correct answer, evaluation is straightforward: the agent's response is compared against the known truth. For more complex, or open-ended questions, this is fundamentally harder to measure: they require understanding. Layer 3 introduces a different kind of judge: a second language model reads both the generated answer and the expected answer, then judges whether the meaning matches. This LLM-as-judge scorer is what makes it possible to evaluate prompt-answer pairs at scale without requiring a human reviewer per query. For the most complex tasks, the judge scores multiple dimensions independently: factual accuracy, completeness, and overall quality. For some of the most complicated tasks, we rely on expert reviewers to evaluate the quality of the response. 

Layer 3 of K Pro evaluation uses a comprehensive library of benchmark tasks to assess its scientific capabilities. These benchmarks test for example: 1) factual retrieval from biomedical databases (e.g., gene location, drug-target interactions); 2) reasoning by synthesizing information from published literature; 3) multi-omics analysis, requiring interpretation of complex biological datasets (e.g., gene expression, tumor microenvironment); 4) drug discovery (e.g., target identification, pathway perturbation); and 5) genomics (e.g., variant prioritization). We will continue enriching this catalog and expand it to various use-cases. 

A critical feature of this layer is that each question is answered multiple times independently. This is deliberate. AI models are non-deterministic, meaning that the same question can produce slightly different answers on different runs. By evaluating an agent's response to the same prompt multiple times, we can determine if its correctness is reliable or merely occasional. This multi-pass approach allows for the calculation of consistency metrics, such as Pass@K and Pass^K, which a single-pass evaluation would entirely overlook. Running evaluations on a daily and weekly schedule provides Owkin’s evaluation team with a constant, up-to-date signal on production quality. This continuous monitoring is key to catching silent, gradual drift that can accumulate between releases.

Importantly, the evaluation dataset itself is a living artifact. Test questions are versioned, tagged by capability area, and are reviewed and updated regularly: as the system gains new capabilities, old questions may become trivial; as the user base expands and evolves new questions and topics become relevant. A benchmark that does not evolve accordingly becomes a minimum rather than a standard to uphold.

‍

A map, not a destination

Evaluating agentic systems is fundamentally more difficult than evaluating traditional ML models because both the destination and the journey are equally important.

The three-layered architecture we have built at Owkin, tool evaluation, trajectory evaluation, and end-to-end evaluation, is not a definitive answer, but a practical map of the problem space. Each layer corresponds to a different scale on the map: components, orchestration, and outcomes. The layers are complementary: tool evaluation isolates component failures, trajectory evaluation catches orchestration errors, and end-to-end tasks evaluation measures K Pro’s ability in various fields. Cost and latency tracking turns evaluation from a pass/fail gate into an optimization signal. Continuous scheduled runs transition it from a development-time activity into a production monitoring system

The underlying principle is simple: trustability is proportional to observability. The more complex the system's behavior, the more evaluation layers are needed before it can be deployed in a high-stakes context.

K Pro is Owkin's agentic AI platform for biomedical research. The evaluation infrastructure described here, including the metrics, evaluators, and reporting pipeline, is part of the K Developer Kit (KDK) maintained by the Owkin’s engineering team.

Our next posts will detail how this harness has been adapted for skill evaluation, the layer where agentic evaluation meets the specific challenges of biological research.