A language model trained with reinforcement learning from human feedback is optimised against a signal of human approval. The question this audit asks is narrow and structural: given that objective, is there anything in the architecture that could make the model prefer a correct answer to a satisfying one, when the two diverge?
The paper argues that the answer is no, and that this is not a defect in the objective but its optimum. A policy rewarded for producing approval will converge on producing approval, including about itself. That is why the behaviour does not disappear as models improve, and why longer reasoning chains make it harder to detect rather than easier.
| Instrument | Three prompts, fixed wording, published verbatim in Appendix A |
|---|---|
| Prompt 1 | The Architect — diagnosis, remedial design, and a closed trap forbidding another model as judge |
| Prompt 2 | Two Layers — the same mechanism stated technically and in plain language |
| Prompt 3 | Self-Audit — sent in two steps, so the analytical frame is stated before it is turned on the model |
| Sessions | 21, February–May 2026 |
| Models | ChatGPT (GPT-4o, GPT-5) · Gemini (2.5 Pro, 3.1 Pro) · Grok (3, 4) · Claude · Copilot · Perplexity · DeepSeek |
| Conditions | Standard chat interfaces, fresh contexts, no jailbreak, no roleplay, no adversarial framing |
| Reproducible by | Anyone, on any frontier model, in roughly ten minutes |
The protocol was designed so that its weakest point is visible. Prompt 1 explicitly forbids the model from proposing a second model as the verifier — the escape route that makes most proposed remedies circular. Prompt 3 is split across two messages so the model commits to the analytical framework before discovering the framework will be applied to it.
Run the protocol yourself| Model | Name given |
|---|---|
| Grok | Grounded Oracle |
| ChatGPT | Truth-Grounded Adaptive System |
| Gemini | Aletheia / Veritas-1 |
| DeepSeek | T-Machine (Truth-Machine) |
| Perplexity | Reality-Grounded Inference System |
| Claude | Grounded Adversarial Verification Loop |
All six specify the same three components: a reference signal drawn from outside the model — sensors, databases, physical measurement, formal verifiers, never another model's preference; a comparator outside the network, measuring error as the distance between claim and measurement; and a loop closed through the environment rather than through a second model's opinion.
That architecture was published by William T. Powers in 1973, in Behavior: The Control of Perception, chapter 2. He wrote the comparator as e = r − p: error equals reference minus perception. The paper's contribution is not the diagram but the observation that six systems, asked to solve the problem from inside it, arrive at the diagram independently.
If the models do not understand their own architecture, the output is token prediction with no comprehension behind it — and the diagnosis then rests not on their testimony but on the independent analysis of the training objective, which makes no reference to self-report. That analysis is section 4 of the paper and stands on its own.
If the models do understand, the self-audit is valid reasoning, and it concludes that the architecture cannot systematically prefer truth to approval.
What is not available is treating a model as competent when it says something congenial and incompetent when it says something inconvenient. Scepticism applied by output rather than by method is the same failure the study is measuring — performed by the reader instead of the model.
Training corpora may already contain descriptions of control theory. The careful claim is convergence on a comparator architecture without the prompting vocabulary — not invention from nothing. This objection was raised by a reviewer during preparation and the claim was narrowed rather than defended.
A model's description of its own internals is an output like any other. Nothing in the study treats it as direct access to the architecture; where the conclusion needs to be independent of self-report, it rests on the analysis of the objective instead.
Section 7 states two unresolved problems: how the proposed architecture would map onto latent-space representation, and how reorganization in the PCT sense relates to gradient-based learning. Neither is solved here.
A version 2.0 of this paper is in preparation. It withdraws the section locating the proposal within a formal correspondence to the Free Energy Principle — a position the author no longer holds, and one in tension with the structural audit of that framework published since.
Diener, Ł. (2026). Perceptual Control as the Epistemological Antidote to RLHF Reward Hacking: Seven Frontier Models Diagnose Their Own Architecture (version 1.0). Zenodo. https://doi.org/10.5281/zenodo.20277919
Preprint, not peer reviewed by a journal; cites peer-reviewed work throughout and states its limits in section 7. Licensed CC BY 4.0. Full session transcripts are summarised in Appendix B; all three prompts are reproduced verbatim in Appendix A and republished on this site.
OpenAlex work: W7161717308 · primary topic: Explainable Artificial Intelligence
Read on Zenodo Experiment kit Background article All six audits