Independence
独立
A reform succeeds, and the institutions that drove it become the maintainers of the new norms. Can they still question this new set of premises independently? This paper says that independence is a relation of a questioner to a set of premises; the same institution can be independent of one set of premises and not of another (S40). Once the premises or duties change, the previous round's conclusion on independence cannot be carried over directly, and the existing evidence must be checked against the new relation (S41). If a questioner's appointment and removal, resources, credentials or evaluation are conditional on maintaining this premise, or the party under review can on its own seal off review of it, the corresponding condition of independence does not hold; implementing, maintaining and measuring are only signals that start a check (S42). Internal division of labor cannot by itself restore independence; it is checked by function (S43). Shared training, credentials or membership are not by themselves enough to prove a lack of independence (S44). These changes are endogenous, not capture; an effective existing separation can be a counterexample to the strong assertion; the success of questioning only reopens the question of independence for the next round (S45). The paper ends with a contrast with the principles of independence in auditing. Six propositions in all (S40 to S45), stated at the end of each section and listed together at the end.
Writing Declaration: This paper was co-drafted with Claude (Anthropic). All intellectual decisions, framework design, and final editorial judgments were made by the author.
Introduction
In the 2010s, a group of researchers in psychology questioned the field's methodological premises: the threshold of significance, choosing analyses after the fact, publishing only "positive" results. The questioning had institutional results. A set of Transparency and Openness Promotion guidelines gave journals a set of standards that can be adopted at graded levels: whether data must be made public, whether analyses must be preregistered, and so on (TOP Guidelines). A form of article called the "Registered Report" was adopted by many journals: the study protocol is peer-reviewed first, and once it passes review, the journal gives an in-principle acceptance, and the study, completed according to the protocol, is published whatever the results.
The institutions of the reform then became the maintainers of these new norms: they maintain the guidelines, revise them, and rate journals' policies. This is a natural result of the reform's success, and no one's fault.
But suppose someone now asks: is preregistration inappropriate in some kinds of research? Has in-principle acceptance brought new biases? What is being asked about are the new premises. At this point, those most familiar with these premises are the institutions that maintain them; their continued existence and reputation are also tied to these premises. Can they still question this new set of premises independently?
This paper does not answer "yes" or "no." What it says is how this question is to be asked, and what must be checked.
The reading follows the Prequel. This paper concerns the second domain, standards that act on persons (Prequel §5). §1 says that independence is a relation; §2, that it cannot be carried over directly; §3 gives the conditions of dependence; §4 speaks of internal division of labor; §5, of membership; §6, of endogeneity and counterexamples; §7 lists several cases to be checked; §8 draws a contrast with auditing.
§1 Independence Is a Relation
To say that an institution is "independent," one must ask: relative to what?
Independence is a relation of a questioner to a particular set of premises. An institution can be independent of premises it never took part in setting; of premises it set itself, maintains, and owes its name to, it may not be. The same institution can be independent of one set of premises and not of another.
So the sentence "this institution is independent," if it does not state relative to which set of premises, is not yet finished. Independence among the four conditions of Paper 2 is indexed in the same way: its continued existence and evaluation, including resources, appointment and removal, credentials and reputation, do not depend on this set of premises continuing to hold (Paper 2, S12). The words "this set" are the relation.
S40 Independence is a relation. Independence is a relation of a questioner to a particular set of premises; the same institution can be independent of one set of premises and not of another.
§2 It Cannot Be Carried Over Directly
Since independence is a relation, when the relation changes, the conclusion must be checked again.
The premises change: a reform changes p into p′. The duties change: the questioner turns from the party questioning p into the party maintaining p′. If either change occurs, the previous round's conclusion that "this institution is independent of p" cannot be carried over directly to p′.
That it cannot be carried over directly does not mean the conclusion has necessarily changed. The existing evidence, protections on appointment and removal, funding arrangements, powers over the agenda, the right to publish reports, is all still there. What must be done is to check this evidence item by item against the new relation: which of it still holds for p′, and which is no longer enough. A protection on appointment and removal may guard against removal from outside, but not necessarily against an evaluation of the kind "not maintaining p′ well enough."
This follows from the previous proposition: since independence is a relation to a set of premises, when the premises change, the relation being checked changes.
S41 It cannot be carried over directly. After the premises or duties change, the previous round's conclusion on independence cannot be carried over directly; the existing evidence (protections on appointment and removal, funding arrangements, powers over the agenda, the right to publish reports) must be checked against the new relation, to settle which of it still holds and which is no longer enough.
§3 Conditions of Dependence
What is to be checked? Here are several observable conditions.
If a questioner's appointment and removal, resources, credentials or evaluation are conditional on maintaining the validity of p, the corresponding condition of independence does not hold. If the party under review can on its own seal off review of p, the condition of independence does not hold either. These two take over from Institution Theory: independence is meant functionally, "what it guards against is the party under review sealing off review of itself on its own" (Institution Theory Z34).
Several things are often taken as proof of a lack of independence, but are really only signals: having implemented p, maintaining p, measuring p. They prompt a check; they are not themselves the result of one. Even among "units that measure p," dependence can differ greatly: a unit commissioned to measure truthfully, for which reporting failure also counts as doing its duty, and a unit whose continued existence depends on proving that the scheme succeeds, are two different things. For the first, measuring "failure" does not threaten it; for the second, measuring "failure" does. The same signal, different dependence.
Once maintaining p′ becomes a condition of the original questioner's continued existence or evaluation, it has become part of the party actually being questioned. Then the sentence from Institution Theory applies as it stands: independence is "checked against the party actually questioned" (Institution Theory Z34). For cases where the party questioned has in fact changed level, for instance where the question has risen from one journal's policy to the norms of a whole field, the second half of the same passage is used: "when the party questioned changes level, independence must be checked again."
S42 Conditions of dependence. If a questioner's appointment and removal, resources, credentials or evaluation are conditional on maintaining the validity of p, or the party under review can on its own close off review of p, then the corresponding condition of independence does not hold. Implementing, maintaining and measuring are signals that start a check, not the result of the check. Once maintaining p′ becomes a condition of the original questioner's continued existence or evaluation, it has become part of the party actually being questioned, and independence is checked against the party actually questioned; when the party questioned changes level, independence must be checked again.
§4 Internal Division of Labor
An institution can divide its labor internally: one group maintains p′, another group evaluates p′. Can this restore independence?
The division of labor alone cannot. It must be checked by function: who can remove the evaluators? Who can limit their resources? Who can block their agenda? Who can suppress their results? Who can, alone, declare that "the evaluation is finished"? If all of these are in the hands of the side that maintains p′, then however the internal group is separated, it is not independent.
Conversely, an internal position can obtain the corresponding protections: a term of office, independent funding, the right to publish reports. An external institution, too, may be under the sway of the same controller. So whether something belongs organizationally inside or outside does not by itself decide independence. Institution Theory says: "independence is functional and does not require being outside the institution" (Institution Theory Z34).
A research note from the Federal Reserve offers corroborating evidence. It discusses the gains and losses of involving economic analysts in drafting rules: involvement can improve the rules, but may weaken the independence of evaluation from policymaking; it cites the practice of the Securities and Exchange Commission, which has different analysts responsible for drafting and for evaluation, to ease this problem; and in a footnote it cautions that "both sets of analysts ultimately answer to SEC leadership," so the concern remains (Kay & Migueis 2025, footnote 16). What this corroboration supports is that "separating the staff is not enough to certify independence automatically"; it does not prove that "functional arrangements within one institution cannot restore independence."
The code of ethics of the accounting profession has a provision in the same direction: the previous work of others in the same firm also falls within the scope of the self-review threat; when facts or circumstances change, the threats must be re-evaluated (IESBA Code, 120.6 A3(b), R120.9).
Recusal is a different matter. A mathematics journal publishing a long paper by its own editor-in-chief stated in the preface to the special issue that the editor-in-chief took no part in handling these manuscripts, "to avoid the conflict of interest that arises because the author is Editor-in-Chief of PRIMS" (PRIMS 57, 2021, Preface, p.1). Recusal deals with one person's conflict of interest in one matter; the independence this paper speaks of is a relation to a set of premises. Both are needed, and they are checked separately.
S43 Internal division of labor cannot by itself restore independence. It is checked by function, according to the four conditions and Institution Theory Z34: who can remove the evaluators, limit resources, block the agenda, suppress results, alone declare a case closed. Internal positions can obtain corresponding protections, and external institutions may also be under the sway of the same controller; organizational belonging does not by itself decide independence.
§5 Membership
Paper 3 left a question: if the members of an evidence review have all received the same training, all hold the same credentials, all belong to the same society, can they still be independent of the discipline's premises?
Shared training, credentials or membership are not by themselves enough to prove a lack of independence. People who received the same training can perfectly well question that training; in the history of the disciplines, premises have mostly been rewritten by people within the discipline.
The condition of lacking independence must rest on dependence: if members' continued existence or evaluation is conditional on p continuing to hold, for instance if maintaining one's credentials is conditional on not denying p, or the evaluation for promotion is conditional on p continuing to be accepted as valid, then the corresponding condition of independence does not hold. Merely being required to follow the current p in one's work is not the same as being conditional on p continuing to hold: a person can work by the current rules while having the right to report truthfully that they fail. Other kinds of dependence, and whether review can be sealed off, are checked separately as in §3.
This answers the question in Paper 3, §5: keeping the membership condition "may not deny p" does not supply independent questioning; and merely "being members together" does not amount to having such a condition.
S44 Membership is not by itself enough to prove a lack of independence. Shared training, credentials or membership are not by themselves enough to prove a lack of independence from p. If members' continued existence or evaluation is conditional on p continuing to hold, the corresponding condition of independence does not hold; other kinds of dependence, and arrangements for closing off review, are checked separately as in S42. This answers Paper 3, S19.
§6 Endogeneity and Counterexamples
The loss of independence this paper speaks of is different from "capture." Capture means the party under review has assimilated the reviewer: the regulated industry has influenced the regulator (Stigler 1971), or has brought the regulator to identify with it culturally (Kwak 2014). Here no outside influence is needed. Reformers becoming maintainers is an internal change in roles and relations of dependence: no one comes to buy anyone off, no one comes to assimilate anyone; it is the success of questioning that has put the questioners in the position of maintainers.
So these changes can come with the success of reform. But success itself is neither sufficient nor necessary. Not sufficient: in a successful reform, if maintenance is handed to another, independent institution, the reformers' independence is not necessarily harmed. Not necessary: the conditions of dependence in §3 can already hold before a reform has taken effect. Suppose a questioner has already turned into the party drafting and maintaining a rewriting that no one has yet adopted anywhere; if its funding or credentials are already conditional on this rewriting continuing to be accepted as valid, so that reporting truthfully that it fails would cost it resources or credentials, then the corresponding dependence is already there, whether or not the reform has succeeded. What is to be looked at is whether roles and relations of dependence have changed, not how far the reform has succeeded.
A strong assertion must be guarded against. "Having taken part in implementation or maintenance, one has lost independence" is too strong. An effective existing separation of functions can refute it: within an institution, if the group that maintains and the group that evaluates are effectively separated in appointment and removal, resources, agenda and results, that is a counterexample. Whether it really blocks the dependence listed in §3 is checked item by item. Whether terms of office, rotation and separation are effective is also checked by function as in §4, and they are not all relabeled as "reconstruction."
A last point: the success of questioning is not enough to guarantee that the previous round's conclusion on independence can be carried over directly; it only reopens the question of independence for the next round. Institution Theory has a sentence in the same direction: an institution "may not close off the conditions of questioning and correction on the grounds that it claims to cultivate, that it has already reformed, or that its aims are right" (Institution Theory Z37). Having reformed is no reason to be questioned no longer.
S45 Endogenous, not capture; an effective separation can be a counterexample. Capture means the party under review has assimilated the reviewer; here no outside influence is needed, it is an internal change in roles and relations of dependence. Success is neither sufficient nor necessary (following from S42); what matters is the change in roles and dependence. An effective existing separation of functions can refute the strong assertion that "taking part in implementation or maintenance means losing independence"; whether it blocks the dependence listed in S42 is settled item by item. The success of questioning is not enough to guarantee that the previous round's conclusion on independence can be carried over directly; it only reopens the question of independence for the next round.
§7 Cases to Be Checked
This section lists several cases; what it writes is what must be checked, not conclusions.
Two periods of the psychiatric diagnostic manual. The third edition of 1980 rewrote the premises of classification (Introduction to Paper 5). From 2002 to 2013, the research agenda touched categorical classification, and the fifth edition did not rewrite the main body (Paper 2, §6). In both periods, whether the revision working groups were independent of the premises they were revising requires separate evidence: how the members' appointment and removal, funding and evaluation relate to this set of premises. It is not inferred backward from "changed" or "not changed."
Evidence-based medicine. What evidence-based medicine questioned in its day was practicing medicine on experience and authority; its institutions have since maintained the hierarchy of evidence and the methods of appraisal. The 2014 article by Greenhalgh and colleagues set out several of its predicaments from within the movement (Greenhalgh et al. 2014). What must be checked is: for review of the premise of the evidence hierarchy by the institutions that maintain it, who initiates it, and who can close it.
The credibility revolution. Paper 2 recorded it as a possible case of "in force but not occupied." Here the conditions of §3 are not tested by "no new institution was set up"; one must trace the specific premise, those who bear it, and how the powers of journals and societies changed accordingly.
Open science in psychology. The example in the Introduction. What must be checked is: whether the funding, evaluation and reputation of the institutions that maintain the guidelines and ratings are conditional on these new norms continuing to hold; and whether review of these norms can be initiated by others.
§8 With Auditing
Auditing has long had principles of independence. The code of ethics of the accounting profession speaks of the self-review threat: when auditors evaluate their own previous work, or the previous work of others in the same firm, there is such a threat; the code also requires threats to be re-evaluated when facts or circumstances change (IESBA Code, 120.6 A3(b), R120.9). The evaluation principles of the OECD Development Assistance Committee say that evaluation should be functionally independent of policymaking and of the delivery and management of aid (OECD DAC 1991, paragraph 11).
So "the unit of independence is the institution" and "when the relation changes, check again" are neither of them this series' contribution. What this paper proposes to advance is two points. First, applying this principle to the constitutive premises of disciplines: what is evaluated is not a set of accounts or a project, but the premises on which a discipline runs. Second, giving observable conditions of change in roles and dependence (§3). One further question is raised, not as a contribution: whether the rules of disciplinary governance contain safeguards like those of auditing, rotation, cooling-off periods, prohibitions on holding dual roles. This paper has not set out a scope of search, so it raises this only as a question.
This paper does not claim that no one has said these two points before.
Where This Paper Stops
This paper does not judge that any particular institution has lost its independence; it does not answer who ought to occupy the questioning position.
What it hands to Paper 9: how a position that cannot sit fixed should be set up, so that it has teeth and yet does not take the discipline's decisions for it.
The remainder it leaves: the four cases in §7 list only the items to be checked; whether disciplinary governance has safeguards like those of auditing is left as a question.
The Six Propositions of This Paper
- S40 Independence is a relation. Relative to a particular set of premises; the same institution can be independent of one set and not of another. (Conceptual distinction)
- S41 It cannot be carried over directly. After the premises or duties change, the previous round's conclusion is not carried over directly; the existing evidence is checked against the new relation. (Conceptual distinction; following from S40, this series' own)
- S42 Conditions of dependence. If appointment and removal, resources, credentials or evaluation are conditional on maintaining p, or the party under review can on its own close off review, it does not hold; implementing, maintaining and measuring are signals; when the original questioner is conditional on maintaining p′, it has become part of the party questioned. (Conceptual distinction: conditions of dependence; takes over Institution Theory Z34)
- S43 Internal division of labor cannot by itself restore independence. Checked by function; organizational belonging does not by itself decide independence. (Conceptual distinction; takes over Institution Theory Z34, "independence is functional and does not require being outside the institution")
- S44 Membership is not by itself enough to prove a lack of independence. Not independent when continued existence or evaluation is conditional on p; the rest checked separately as in S42; answers Paper 3, S19. (Conceptual distinction)
- S45 Endogenous, not capture; an effective separation can be a counterexample. An internal change in roles and dependence; success neither sufficient nor necessary; an effective separation can refute the strong assertion, checked item by item; success only reopens the question for the next round. (Conceptual distinction; "success is not necessary" follows from S42; the reading points the same way as Institution Theory Z37)
References
All SAE works below are cited by concept DOI, which always resolves to the latest version.
I · This system
- Qin, H. (2026). Open to Questioning (SAE Institution Theory, Paper 6) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23271324
- Qin, H. (2026). Standard-Setting (SAE Theory of Disciplines, Paper 2) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.24177754
- Qin, H. (2026). Misrouting (SAE Theory of Disciplines, Paper 3) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.24177759
II · Rule texts and institutional documents
- Publications of the Research Institute for Mathematical Sciences, 57 (2021). Preface to the special issue, p.1.
- Center for Open Science. Transparency and Openness Promotion (TOP) Guidelines (current version); description of Registered Reports.
- International Ethics Standards Board for Accountants. (2024). Handbook of the International Code of Ethics for Professional Accountants (2024 edition), 120.6 A3(b), R120.9.
- OECD Development Assistance Committee. (1991). Principles for Evaluation of Development Assistance.
III · External literature
- Greenhalgh, T., Howick, J., & Maskrey, N. (2014). Evidence based medicine: A movement in crisis? BMJ, 348, g3725.
- Kay, B. S., & Migueis, M. (2025, June 26). How to design rules for ex-post evaluation. FEDS Notes. Board of Governors of the Federal Reserve System.
- Kwak, J. (2014). Cultural capture and the financial crisis. In D. Carpenter & D. A. Moss (Eds.), Preventing Regulatory Capture. Cambridge University Press.
- Stigler, G. J. (1971). The theory of economic regulation. The Bell Journal of Economics and Management Science, 2(1), 3–21.
Interfaces
Main upstream: Institution Theory Z34, Z37; Paper 2, S12; Paper 3, S19.
Also used: the TOP Guidelines and the rules for Registered Reports; the preface to the PRIMS 57 special issue; Kay & Migueis 2025; the IESBA Code; OECD DAC 1991; Greenhalgh et al. 2014; Stigler 1971; Kwak 2014.
This series: S40 to S45.
Handed to Paper 9: how a position that cannot sit fixed should be set up.