proof42.ai / Articles / Pattern
Pre-Registering AI Output Tests So the Results Hold Up
A pre-registered, cross-platform study reveals why post-hoc testing fails and how protocol-first design builds audit-ready AI verification.
Why Post-Hoc AI Tests Mislead Decision Makers
Testing AI output after the fact introduces hidden bias that invalidates the results. When you adjust your criteria based on what the model produced, you create a moving target that favors the outcome you wanted to see. This practice, known as p-hacking in statistics, makes it impossible for others to verify your claims or replicate your findings.
Post-hoc analysis often misses critical failure modes because the test was not designed to catch them. Without a pre-defined plan, you might overlook subtle issues like content misattribution where a URL exists but points to the wrong entity. These errors remain invisible until a specific validation step is added, which is too late for a rigorous audit.
Organizations relying on ad-hoc tests cannot defend their AI outputs in high-stakes environments like law or medicine. Regulators and auditors require evidence that the testing process itself was robust and unbiased. A test designed after the results are known lacks the structural integrity needed to support liability decisions or safety certifications.
The solution lies in shifting the focus from the result to the process. By committing to a protocol before data collection begins, you force yourself to consider all potential failure points upfront. This discipline ensures that the evaluation set covers the full range of risks, not just the obvious ones that happen to appear in a casual trial.
Writing the Protocol Before the First Run
A valid test starts with a written protocol that specifies the exact prompts, platforms, and metrics to be used. You must define the primary outcome, such as the rate of fabricated URLs, and the statistical methods for analyzing it before generating any responses. This pre-registration prevents you from cherry-picking favorable data points later.
The protocol should include a clear definition of what constitutes a failure. For source fabrication, this means distinguishing between a URL that does not exist and one that exists but contains incorrect information. Your validation steps must be automated or strictly defined to remove human judgment from the scoring process.
You must also decide on the scope of the test, including the number of entities to probe and the depth of the conversation. A fixed number of turns, such as six, ensures that the test simulates realistic usage without drifting into infinite loops. Every variable in the experiment must be locked down to prevent drift during execution.
Documenting the protocol on a public repository adds a layer of accountability. It allows third parties to see exactly what you planned to do before you knew the results. This transparency is essential for building trust in AI systems, especially when the stakes involve public safety or financial liability.
Factorial Design Across Platforms and Conversation Depths
A factorial design allows you to test multiple variables simultaneously, such as the AI platform and the style of the user prompt. By crossing these factors, you can determine if a specific prompt style works differently on different architectures. This approach reveals interactions that a simple A/B test would miss.
Conversation depth is a critical variable in AI testing because errors often compound over time. You should test how fabrication rates change from the first turn to the sixth turn. This reveals whether the model gets worse as the dialogue continues or if it stabilizes after an initial error.
Using fictional entities as test subjects eliminates the confusion of real-world ground truth. When you query a company that does not exist, any citation the model provides is definitionally a fabrication. This method provides a clean signal that is not obscured by the varying quality of real-world data.
Running the same test on both parametric and retrieval-grounded systems highlights architectural differences. Parametric models may invent URLs that look real but return errors, while retrieval systems may link to real pages with the wrong content. A factorial design captures both failure modes in a single study framework.
Documenting Deviations Openly to Maintain Integrity
Even with a perfect plan, technical issues will arise during data collection. You must record every deviation from the original protocol, such as a software bug that failed to run a validation check. Hiding these errors undermines the credibility of the entire study.
Transparency about deviations allows readers to assess the impact on the results. If a validation step failed for half the dataset, you can explain how you corrected it or why the remaining data is still valid. This honesty builds more trust than a flawless-looking report that hides its flaws.
Some deviations may require statistical adjustments, such as switching to a different model when data is sparse. You should document the rationale for these changes and show that they do not alter the core conclusions. The goal is to show that the results are robust despite the imperfections in execution.
A public log of deviations serves as a case study for future researchers. It teaches the community how to handle unexpected problems without compromising scientific rigor. This shared knowledge helps organizations build better testing pipelines for their own AI systems.
Reporting Reversed Hypotheses as Valid Findings
Scientific progress depends on reporting results that contradict your initial expectations. If your hypothesis that rejection framing increases fabrication is proven false, you must report that clearly. A null result is still a valuable finding that prevents others from repeating the same mistake.
In the study of AI source fabrication, the data showed that challenging the model did not increase errors. Instead, the baseline requests produced the highest fabrication rates. This counter-intuitive result changes how organizations should interact with AI systems to minimize risks.
Reporting reversed hypotheses requires confidence in your methodology. If you are sure your protocol was sound, then the data speaks for itself. You do not need to force the results to fit a story; the truth of the data is sufficient for decision-making.
This approach shifts the culture of AI testing from confirmation to discovery. Organizations stop looking for proof that their system is perfect and start looking for evidence of where it fails. This mindset leads to more robust systems and safer deployments in the real world.
Key takeaways
- Pre-registering your test protocol before collecting data prevents bias and ensures results are auditable.
- Factorial designs across platforms and conversation depths reveal failure modes that simple tests miss.
- Documenting every deviation from the plan openly maintains the integrity of the research.
- Reporting hypotheses that are disproven by the data is a valid and necessary part of scientific inquiry.
- Using fictional entities as probes provides a clean ground truth for measuring source fabrication rates.
Related questions
What is the main benefit of pre-registering an AI output test?
Pre-registration locks in the methodology before data collection, preventing you from adjusting criteria to match desired results. This ensures the test is objective and the findings can be independently verified by auditors or regulators.
How does a factorial design improve AI testing compared to A/B testing?
A factorial design tests multiple variables, like platform type and prompt style, simultaneously. This reveals interactions between factors, showing how different architectures respond to specific user behaviors that a simple two-group test would miss.
Why should I use fictional entities instead of real ones for testing?
Fictional entities have no real online presence, so any citation the model provides is definitively a fabrication. This removes ambiguity about ground truth and allows for precise measurement of error rates without the noise of real-world data variations.
AI-assisted: drafted with a language model from the author's research notes, then edited and checked before publication. Author: Zoltán Varga, AI visibility researcher and AI ecosystem architect, Budapest.
