The Rule Passed Its Tests. What Did the Tests Establish?

Compile-Time Intelligence, Part 2

Imagine being able to take a clinical guideline, an insurance policy, or a complex legal agreement and turn its requirements into software that applies them consistently, thousands or even millions of times.

The potential value is substantial: written knowledge becomes something an organization can put to work, inspect, and update. But consistency cuts both ways. A missing exception or misinterpreted threshold could also become a repeatable mistake.

Part 1, AI Is Discovering New Mathematics. The Hard Part Is Knowing Exactly What It Proved, explored the distinction between checking a formal statement and establishing that it captures what humans intended. A proof can be valid while the translation of the original question remains wrong. The same distinction matters when a policy becomes executable logic. [1]

Testing seems like the practical answer. Give the rule examples, specify the expected results, and see whether they match.

But a passing suite can leave a consequential mistake untouched, not because the tests malfunctioned, but because they never asked the question that would expose it.

Part 1 asked whether we formalized the right thing. This post asks whether our tests would reveal when we did not.

A rule that passes its tests

Consider a fictional clinical cohort definition. A patient belongs to the cohort, the group being selected, if any one of three qualifying signals is present: a diagnosis, a medication, or a laboratory result above a specified threshold. A specified exclusion overrides all three.

For now, each fact has already been resolved as yes or no. A case is a particular patient scenario. Its inputs are the four values supplied to the software.

A model translates the definition into a rule, executable logic that takes those inputs and produces an inclusion decision:

in_cohort = (
    not excluded
    and diagnosis
    and (medication or lab_above_threshold)
)

Here, and requires both connected conditions, or requires at least one, and not excluded requires the exclusion to be absent.

The model also supplies three tests. Each runs the rule on a case and compares its answer with an expected result: a patient with all three signals and no exclusion should be included; a patient with none should be excluded; and a patient with all three signals and the exclusion should also be excluded.

All three tests pass.

Now add a patient with a qualifying diagnosis, no medication, no qualifying laboratory result, and no exclusion. The source definition says this patient belongs. The generated rule leaves them out.

The model made diagnosis compulsory and required additional evidence, although the source allowed each signal to qualify on its own. The original tests never distinguished those two requirements.

What a passing test tells us

A test suite is a collection of tests. In our example, each test compares the rule’s actual result with an expected result.

But who or what determines the expected result?

That is the role of the test oracle: the criterion or mechanism used to judge whether the output is acceptable. For an exact-answer test, it supplies the expected answer for that case. Other oracles check conditions that acceptable behavior must satisfy. [2]

In our example, the oracle can be as simple as a recorded answer justified by the cohort definition: “For a diagnosis-only patient with no exclusion, expect ‘include.’”

Two things can therefore go wrong. The cases may miss the situations where the rule fails, or the oracle may supply incorrect expectations.

For the moment, assume our expectations faithfully reflect the fictional definition. Even with that assumption, the original suite missed an error.

What makes a better set of cases?

Find the cases that separate the rules

A useful way to picture a rule-based program is as a logical decision tree. As it checks conditions, a case takes different directions depending on the answers. The alternatives at a decision point are branches; a sequence of steps through the program is an execution path. Real programs can also merge paths or repeat steps, so the tree is a simplified picture. Code coverage measures which parts of the implementation the tests exercise: line coverage tracks which executable lines ran, while branch coverage tracks which alternatives at decision points were taken. Neither means that every complete path or combination of inputs was tested. [3]

Our fictional definition gives patients three independent qualifying routes, three ways to satisfy the inclusion requirements. Diagnosis alone is enough. Medication alone is enough. A qualifying laboratory result alone is enough. Each requires the exclusion to be absent.

That suggests three revealing tests:

The rule fails all three.

A patient with every signal cannot distinguish “any one is enough” from “diagnosis plus another signal is required.” Both rules include that patient. Removing the other signals reveals which requirement the code actually implements.

These cases came from examining the source, not merely the implementation’s branches. A qualifying route describes what the requirements allow. A code branch describes how the program executes.

A coverage report can show that existing code ran. It cannot, by itself, establish that the implementation contains every requirement it was supposed to contain. Even complete branch coverage does not guarantee that the tests exercised the particular input combinations that would reveal an error. [3] [4]

This is why the choice of cases matters. The diagnosis-only patient does more than execute the program again: it checks a requirement the original tests never isolated. A stronger suite needs cases that make incorrect behavior observable, not merely a larger count of executions.

Research on generated code demonstrates how much this can change an evaluation. HumanEval is a collection of programming tasks with tests for proposed solutions. EvalPlus expanded those tests roughly 80-fold using model-proposed inputs and systematically generated variations (Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang).

In its 2023 evaluation, GPT-4’s task pass rate fell from 88.4% under the original tests to 76.2% under the expanded suite. The generated programs had not changed. The stronger evaluation exposed failures the original tests had missed. [4]

That raises a practical question: how can AI help find cases that a simpler test-generation approach overlooks?

Examining the source is one approach. Examining the implementation is another. If an exclusion check exists in the code but no test reaches it with an excluded patient, inspecting that code can suggest a case that exercises the neglected behavior.

SymPrompt takes this code-aware approach. It guides test generation using execution paths and the context needed to call the code correctly, such as required imports and objects (Gabriel Ryan and colleagues). In its GPT-4 experiment, average line coverage increased from 36% under the baseline prompting approach to 74%. More of the implementation was exercised; the result does not measure a corresponding improvement in bug detection or fidelity to the source. [5]

These perspectives complement one another. The implementation helps identify behavior that existing tests have not exercised. The source helps identify behavior that should exist, including requirements the implementation may have omitted.

Coverage can therefore guide test generation without becoming the definition of correctness.

Test whether the tests can catch an error

Those approaches help find additional cases. But how can we assess whether a suite is actually capable of detecting particular mistakes?

One approach is to deliberately alter the program and see whether the tests notice.

After repairing our example, the intended rule is:

in_cohort = (
    not excluded
    and (diagnosis or medication or lab_above_threshold)
)

Mutation testing creates altered copies of a program, called mutants, and runs the tests against them. A mutant is detected when at least one test rejects its behavior. It survives when the suite does not detect the alteration. [2]

Earlier, we varied the inputs. Here, we change the implementation while keeping the cases and expectations fixed.

Consider three separate alterations: restore the requirement for diagnosis plus another signal, delete the laboratory-only qualifying route, and remove the exclusion.

The original three tests detect the missing exclusion: an otherwise eligible excluded patient would now be admitted. They do not detect either of the other changes.

The additional single-signal cases do. The diagnosis-only case exposes the requirement for additional evidence. The laboratory-only case exposes the missing qualifying route.

That tells us more than “the suite grew from three tests to six.” It identifies mistakes the suite can now detect.

A mutation score summarizes the fraction of evaluated mutants detected by the tests. Its meaning depends on which alterations are evaluated. Some alterations also leave behavior unchanged; these equivalent mutants should not be counted as ordinary missed errors. [2]

For our example, detecting many minor alterations would not compensate for missing the one that removes the exclusion. The informative result is not just a percentage, but an account of which consequential mistakes the suite can distinguish.

Why the distinction matters outside a toy example

A clinical study illustrates why the way a rule combines evidence deserves this attention.

In 2024, Chao Yan and colleagues evaluated model-generated phenotyping algorithms, database queries that identify patients with specified characteristics, against clinician-validated reference algorithms from the eMERGE research network. GPT-4’s dementia query retrieved only 24.4% of the reference cases. The authors attributed its restrictiveness primarily to requiring medication evidence alongside other criteria. [6]

This was a research-cohort evaluation, not a deployed treatment-system trial. It did not evaluate the illustrative mutation process described here. Its relevance is the failure mode: executable code combined evidence in a way that excluded many patients identified by the reference.

Our fictional example makes that mechanism easy to inspect. Four true-or-false inputs have only 16 combinations, so all can be tested.

Real systems have another problem underneath those inputs. Which diagnosis counts? How recent must a laboratory result be? Does “no exclusion” mean it was checked and found absent, or that no information was available?

Testing the final Boolean expression, the logic combining those true-or-false values, does not settle those questions. They determine what the inputs mean and what the expected answers should be.

An analogous mistake in a system that prioritizes clinical review could repeatedly leave intended patients unflagged. That is a possible consequence of the failure mode, not an outcome measured in the study.

A better question than “Did it pass?”

The original three tests were useful for three cases. The mistake was allowing their success to stand in for distinctions they never examined.

A stronger suite can provide more specific evidence: it checks each qualifying route, catches a missing exclusion, and rejects particular restrictive alternatives.

The practical question becomes:

Which plausible mistakes would these tests catch?

That question gives AI-assisted testing a constructive purpose. Models can help generate challenging inputs and explore existing code; source-grounded expectations and deliberate alterations make it possible to examine what those tests actually establish.

But every comparison still depends on its expected answer. A diagnosis-only case fails to expose the bug if the answer key incorrectly says “exclude.”

Part 3 takes up that problem: who or what establishes the expected answers, and how can those expectations be tested and reviewed?

Building a system that turns policies, contracts, or clinical guidance into executable logic? Talk to Eventum about the translation, evaluation, and review process around it.

Sources

1. Bressler D. AI Is Discovering New Mathematics. The Hard Part Is Knowing Exactly What It Proved. Eventum AI, October 13, 2026. Compile-Time Intelligence, Part 1.
2. Hossain SB, Filieri A, Dwyer MB, Elbaum S, Visser W. Neural-Based Test Oracle Generation: A Large-Scale Evaluation and Lessons Learned. ESEC/FSE 2023. DOI: 10.1145/3611643.3616265. Discusses test oracles and evaluates fault detection using mutation testing.
3. Coverage.py. Branch Coverage Measurement. Version 7.16.2 documentation. Explains statement and branch coverage through execution examples.
4. Liu J, Xia CS, Wang Y, Zhang L. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. NeurIPS 2023. Introduces EvalPlus and HumanEval+. Table 3 reports the GPT-4 comparison under greedy decoding; the evaluation also includes corrected reference implementations and explicit input contracts.
5. Ryan G, Jain S, Shang M, Wang S, Ma X, Ramanathan MK, Ray B. Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting Using LLM. FSE 2024. DOI: 10.1145/3643769. Introduces SymPrompt. Table 2 and Section 4.4 report the GPT-4 coverage comparison against a describe-then-generate baseline.
6. Yan C, Ong HH, Grabowska ME, et al. Large Language Models Facilitate the Generation of Electronic Health Record Phenotyping Algorithms. Journal of the American Medical Informatics Association 31(9), 1994-2001 (2024). DOI: 10.1093/jamia/ocae072. Table 2 and the implementation results report the dementia finding.