Misrouting
错送
Paper 2 said that the questioning position can be empty, yet it often seems to have someone in it, because evidence review is always reviewing. This paper asks what the conclusions of a review are taken to have answered. A review has three scopes: what it is authorized to handle; what it actually handled, and what its conclusion can prove; what consequences it is given. The three scopes are registered separately, and a review is not judged by its name (S15). Misrouting is one kind of transfer of use: a challenge is registered as answered by the conclusion of a review that did not in fact handle it, or whose conclusion states a scope that falls short of what it is said to have answered. Misrouting has two directions, sending forward and pushing back, which address different parties; where the answer reached the matter and the outcome was retention, it is not misrouting (S16). Passing a review proves the part actually handled; the end of a procedure is not a substantive answer; consequences beyond the scope of proof are a transfer of use, and a transfer of use that goes as far as registering a challenge as answered is misrouting (S17). Misrouting does not need anyone to fail in duty or to dissemble (S18). Expanding the mandate of evidence review cannot, by that alone, supply independent questioning (S19). Finally, a pathway is set up to be tested: accepting only the conclusions of evidence review as grounds for closing a case, while excluding normative questioning from other procedures (S20). In the public records this paper examined, no individual case of pushing back was found, and the paper says so plainly. Six propositions in all (S15 to S20), stated at the end of each section and listed together at the end.
Writing Declaration: This paper was co-drafted with Claude (Anthropic). All intellectual decisions, framework design, and final editorial judgments were made by the author.
Introduction
In 1972 the physicist Weinberg wrote a short essay about a particular kind of question. His example was low-dose radiation. At very low doses, would radiation raise the mutation rate in mice by a tiny amount, say by half a percent? This is a question of fact, and it can be asked in the language of science; but to answer it by direct experiment at the usual level of confidence would need about eight billion mice: by strict scientific standards it can be conceived of in principle, but in practice it cannot be done (Weinberg 1972, pp.210–211). Weinberg called such questions "trans-scientific": they can be asked of science, but science cannot answer them (Weinberg 1972, p.209). In his example, the reason they cannot be answered is that in practice the work cannot be done.
Borrowing this question of scope, what follows supposes a situation of handover; it is constructed by this paper, not something Weinberg recorded. Suppose experiments at high doses have been done, the data reviewed, and the conclusion is sound; it speaks of the range of doses the experiments covered. Suppose someone asks which model should be used to extrapolate to low doses, and the answer is "science has already reviewed this." The thing reviewed and the thing asked are not the same thing: that the question in fact cannot be answered does not prevent its being registered as answered; a challenge is turned back by "it has already been reviewed." What was reviewed, and what was turned back, this paper records as two separate things.
This paper does not say that evidence review is done badly. The review at high doses can be beyond reproach. What this paper speaks of is where, after the review is done, its conclusion is sent, and what it is taken to have answered.
The reading follows the Prequel. This paper concerns the second domain, standards that act on persons (Prequel §5). §1 gives the three scopes of a review; §2 defines misrouting and gives its two directions; §3 speaks of transfer of use; §4 says that misrouting needs no bad actors; §5 says what an expanded mandate cannot supply; §6 sets up a pathway to be tested; §7 states plainly what was found.
§1 Three Scopes
A review has three scopes, to be registered separately.
-
What it is authorized to handle. Documents of authorization, letters of mandate and terms of reference state what it may review. An evidence assessment panel is authorized to assess the quality of a set of studies; a standards review board is authorized to review the procedure by which a standard was developed.
-
What it actually handled, and what its conclusion can prove. The record of handling states which materials it looked at and which questions it answered; the conclusion states its own scope: under which conditions, and to what extent.
-
What consequences it is given. What the law or an adopter does with the conclusion: admission, rejection, pricing, closing a case.
The three scopes are often of different sizes. What is authorized can be wider than what was actually handled: this time it was not reviewed. What was actually handled can also be wider than what was authorized: it reviewed beyond its bounds, which is acting beyond authority and is checked separately. The consequences given can be far larger than what the conclusion can prove.
One misreading should be headed off here. To say "evidence review" is not to say that it touches only the descriptive ledger and never the normative ledger. Evidence assessment can legitimately contain values: how strong the evidence must be to judge whether a substance is toxic depends on the consequences of wrongly judging that it is and wrongly judging that it is not, and this weighing is itself a matter of values (Douglas 2000 discussed this, following Hempel in calling it inductive risk; Hempel 1965). So whether a review reviews a certain line on the normative ledger cannot be judged by the name "evidence seat"; it is judged by the text of authorization and the actual record of handling.
The 1983 report of the US National Research Council on risk assessment (commonly called the Red Book) called on agencies to "establish and maintain a clear conceptual distinction" between assessing risks and considering risk management alternatives; the same report expressly did not recommend moving assessment as a whole into an organization separate from the regulatory agencies (National Research Council 1983). The functions were separated in concept; the seats were not. This paper uses exactly this kind of division: what is divided is functions and scopes, not institutions.
S15 Three scopes. A review has three scopes: what it is authorized to handle; what it actually handled, and what its conclusion can prove; what consequences the law or adopters give to its conclusion. The three scopes are registered separately. Evidence assessment can legitimately contain values; so whether a review reviews a certain line on the normative ledger is not judged by the review's name, but by the text of authorization and the actual record of handling.
§2 Misrouting
Definition. Misrouting is one kind of transfer of use: a challenge to a premise p is registered as answered by the conclusion of a review that did not in fact handle the challenge, or whose conclusion states a scope that falls short of what it is said to have answered. To say "the challenge to p was ended" means the same thing.
This definition compares scopes by text: what the matter asked is, and how far the review's conclusion says it reaches; the two are set side by side. It does not assess whether the evidence review itself was well done.
For a standard to enter a limited review is in itself a normal step. A standard is submitted to procedural review, and procedural review reviews only procedure; there is nothing wrong in that. Misrouting happens at the step where the conclusion is sent out: the conclusion of a procedural review is taken to answer a question it did not handle.
Three common shapes. First, the reviewer had no authority to review this matter, and its conclusion is taken to answer it. Second, the reviewer had the authority, but this time did not review it, and the conclusion is taken to answer it all the same. Third, part of it was reviewed, and the conclusion is then used more widely. The three are not mutually exclusive; two can occur together in one case.
Six fields. So cases are not classified by shape but registered by six fields:
| Field | What is recorded |
|---|---|
| 1. The specific matter asked this time | Two kinds kept apart: judging by p (by this threshold, does this result count as significant); questioning p's source, alternatives or withdrawal (where does this threshold come from, is there another way to set it, should it be withdrawn) |
| 2. Whether the task submitted or mandated includes it | Letter of mandate, agenda, list of review questions |
| 3. Whether there is authority to handle it | Documents of authorization, terms of reference |
| 4. What was actually handled | Record of handling, body of the review report |
| 5. The scope the conclusion states for itself | Conditions and limits stated in the conclusion |
| 6. What consequences it is given | Adoption documents, record of closure, letters of reply |
The first field matters most. For one and the same threshold, "judging by it" and "questioning it" are two things; that a review handled the first does not mean it handled the second.
Take the supposition in the Introduction as an illustration, not as a case. Suppose someone questions where the extrapolation model used at low doses in a radiation protection standard comes from, and whether there are alternatives (field 1: questioning p); suppose an earlier review assessed the quality of experimental data at high doses (field 2: not included); whether it had authority to review the extrapolation model is checked separately (field 3); suppose what it actually handled was dose and effect at high doses (field 4); its conclusion states that it holds within the range of doses the experiments covered (field 5); suppose this conclusion is cited as the established conclusion of science and ends the challenge to the extrapolation model (field 6). Field 1 does not match fields 4 and 5, yet field 6 treats them as matching: this is what misrouting looks like. How the dispute over low-dose extrapolation is handled in reality, this paper has not checked and does not judge.
Two directions. Misrouting happens at a handover, and it has two directions.
Sending forward: the party submitting a standard that contains p sends it to a limited review, and then enlarges the resulting conclusion into "p has been answered."
Pushing back: the party receiving a challenge to p pushes it back onto an existing review that did not handle p, and closes the case on that basis.
The two address different parties: for sending forward, look at how the submitting party uses the conclusion; for pushing back, look at how the receiving party handles the challenge. What Weinberg gives is a type of question: it can be asked of science, and science cannot answer it. It provides an occasion where pushing back easily happens, and it is a near neighbor of distinguishing the scope of what science answers; the chain of handover in which a conclusion within its scope is used to end another challenge is an illustration constructed by this paper, not Weinberg's record. Step 4 of the checking chain in Paper 1, §3 is precisely pushing back.
Contrast. Where the answer has reached p and the outcome is retention, the challenger's disagreement with that outcome is not grounds for a finding of misrouting. Whether the answer reached the matter is decided by whether fields 4 and 5 match field 1, not by whether the challenger is satisfied.
S16 Misrouting and its two directions. Misrouting is one kind of transfer of use: a challenge to p is registered as answered by the conclusion of a review that did not in fact handle the challenge, or whose conclusion states a scope that falls short of what it is said to have answered ("ending the challenge to p" is synonymous). Scopes are compared by text; whether the evidence review itself was good is not assessed. The three common shapes (no authority to review; authority but not reviewed this time; partly reviewed and the conclusion used more widely) are not mutually exclusive, and cases are registered by six fields: the specific matter asked (keeping apart "judging by p" and "questioning p's source, alternatives or withdrawal"); whether the submitted task includes it; whether there is authority to handle it; what was actually handled; the scope the conclusion states for itself; what consequences it is given. Sending forward: the submitting party enlarges the conclusion of a limited review into p's having been answered. Pushing back: the receiving party pushes a challenge to p back onto an existing review that did not handle p, and closes the case on that basis. The two address different parties. Where the answer has reached p and the outcome is retention, it is not judged misrouting.
§3 Transfer of Use
Passing a review proves the part it actually handled. What one compares against is the scope the conclusion states for itself and the record of handling.
Institution Theory has a sentence: "That a demand bears the name of procedure does not prove that complying with it is within the line." (Institution Theory Z12) The original speaks of how an institution's demands are carried out: having done as the rules say does not prove that the line was not crossed. Here its form is borrowed and its object changed: passing in procedure does not prove that the matter was answered in substance. The "line" in the original is the boundary between limit and encroachment; here it is replaced by a line on the normative ledger (own undertaking).
Both ends need guarding. At one end: the end of a procedure is not a substantive answer to the question. That a review has run its course and been stamped does not say which matter it handled. At the other end: that a review's scope is limited does not by itself prove that anyone claimed more than was theirs. A limited scope is the normal state of things; there is something to record only when consequences exceed the scope.
So the definition of transfer of use rests on consequences: where the consequences that adopters or the law give a conclusion exceed its scope of proof, that is transfer of use. Not every transfer of use is misrouting. A conclusion of "no visible effect within the range of experimental doses" being used to set a limit at low doses is transfer of use; if it is at the same time used to declare that "the challenge to the extrapolation model has been answered," the transfer of use has reached the step of misrouting.
Whether there was also action beyond authority is checked separately against the documents of authorization. Acting beyond authority is the reviewer doing what it had no authority to do; transfer of use is others doing, with its conclusion, more than it proves. The two can happen together, or only one of them.
S17 Transfer of use beyond the scope of proof. Passing a review proves the part it actually handled, compared against the scope the conclusion states for itself and the record of handling. The end of a procedure is not a substantive answer to the question; that a review's scope is limited does not by itself prove that anyone claimed more than was theirs. Where the consequences that adopters or the law give the conclusion exceed its scope of proof, that is transfer of use; a transfer of use that goes as far as registering a challenge to p as answered is misrouting. Whether there was also action beyond authority is checked separately against the documents of authorization.
§4 No Bad Actors Needed
Misrouting can happen while every person is carrying out their duties in good faith.
The reviewer reviewed as authorized, reviewed carefully, and stated the scope in its conclusion. The adopter adopted according to the rules, and the rules say "what has passed review may be adopted." The one answering the challenge answered according to practice, and practice says "matters already reviewed are not reviewed again." Every step has a basis, and no one went beyond their duties; yet the matter that was asked was never handled by anyone from beginning to end, and it was registered as answered.
This matters, because it decides where to look. If misrouting depended on someone hiding values in science, what one would look for is motive; if it can happen in the course of good-faith duty, what one looks for is the rules at the handover: how conclusions are sent out, and how challenges are registered.
An actor hiding his own position behind science is a different mechanism. Pielke discussed the "stealth issue advocate": one who, in the posture of only supplying science, in fact advances a policy position (Pielke 2007). For that, what must be checked is the actor; the misrouting this paper speaks of does not wait for it to happen. The two can appear together, and are to be checked separately.
S18 Neither failure of duty nor dissembling. Misrouting can happen while everyone carries out their duties in good faith, without waiting for someone to hide values in science. An actor hiding a position behind science is a different mechanism, checked separately.
§5 An Expanded Mandate Cannot Supply It
A natural thought is: since evidence review does not handle normative questioning, expand its mandate so that it does.
This step is useful, but by itself it cannot supply what is missing. If the mandate is expanded but the membership condition remains "may not deny p," for instance that only those who accept this set of methods can be selected, then independent questioning relative to p is still absent: when it comes to asking whether p should be withdrawn, the questioner's own position is tied to p. Independence, among the four conditions of Paper 2 (S12), is not met at this point. The relation between membership and independence is set out in detail in Paper 8.
There is another line: the inside will change itself. Internal rewriting does happen, and the history of the disciplines has many instances. But this possibility by itself is not enough to show that there is already an addressable arrangement for questioning relative to p. Institution Theory's demand on channels is: "it is independent of the party questioned, internal subjects can actually initiate it, and initiating it brings no retaliation"; independence is about this: "what it guards against is the party under review sealing off review of itself on its own" (Institution Theory Z34). Which conditions hold and which do not are checked separately. Once the arrangements of membership and accountability change, the matter is decided anew.
In Chapter 4 of Science as Social Knowledge, Longino lists several social conditions under which the criticism within a scientific community can change something: recognized venues for criticism; shared standards of evaluation; the community responding to criticism, its views changing in consequence; and equality of intellectual authority among qualified participants (Longino 1990, Chapter 4, pp.76–79). Her point is that peer review can screen out the idiosyncratic biases of individuals but not assumptions shared by the whole community, so objectivity depends on these social conditions. This is close to the question "will the inside change itself." Where they part must be stated: responding to criticism, views having changed, and a position having institutional authority to require a public answer are three things. The first can exist without the third: the response is voluntary. The second can also exist without the third: views changed for other reasons. The right of response spoken of in Paper 2 is the third. How Longino's conditions correspond one by one to these three is not judged here.
S19 Expanding the mandate of evidence review cannot, by itself, supply it. Expanding the mandate while keeping the membership condition "may not deny p" is still not enough to supply independent questioning relative to p (by independence in Paper 2, S12; set out in detail in Paper 8). Internal rewriting does happen; this possibility by itself is not enough to show that there is already an addressable arrangement for questioning relative to p that meets the conditions of Institution Theory Z34. Which conditions hold and which do not are checked separately; once the arrangements of membership and accountability change, the matter is decided anew.
§6 A Pathway to Be Tested
Misrouting comes by many routes. Only one of them is set up here, as a mechanism to be tested, not as the common source of all misrouting.
If an institutional arrangement accepts only the conclusions of evidence review as grounds for closing a case, and also excludes questioning of normative premises from other effective procedures, then normative choices may continue to run without explanation, in the guise of technical conclusions. Both conditions must hold together: closure accepts only evidence review; normative questioning cannot get in elsewhere either. With only the first, normative questioning can still go elsewhere; with only the second, a case can still be closed on other grounds.
This pathway is to be checked from the actual rules of exclusion and closure: how intake rules are written, what grounds closure requires, where rejected matters are referred. It can happen in the course of good-faith duty, and is compatible with S18 in §4.
Compare the European decision in the Introduction to Paper 1: in the formal objection procedure the Commission did not take up claims about installation and use, and at the same time stated that member states may regulate installation and use (Commission Implementing Decision (EU) 2017/1996). The first part resembles the second condition of this pathway, but the second part states where the power lies, so it does not fall on this pathway; it is recorded as "destination of the power stated." In the rules this paper examined, no intake rule was found that has both conditions.
S20 A pathway to be tested. If an institutional arrangement accepts only the conclusions of evidence review as grounds for closing a case, and also excludes questioning of normative premises from other effective procedures, normative choices may continue to run without explanation, in the guise of technical conclusions. This is one possible pathway of misrouting, not the common source of all misrouting, and is to be checked from the actual rules of exclusion and closure; it can happen in the course of good-faith duty, and is compatible with S18.
§7 What Was Found
Stated plainly: in the public records this paper examined (scope in Note 1), no individual case of pushing back was found; the existing material cannot determine why none was found. What was found are three possible sites, and one example to be excluded.
Risk of bias and surrogate endpoints. Evidence assessment in evidence-based medicine has a set of tools for rating risk of bias; what they rate is whether a trial's design and conduct might make its results depart from the truth. In 2014 Greenhalgh and colleagues wrote in the BMJ listing several predicaments of evidence-based medicine. In one place they say that vested interests decide which tests and treatments are compared, and "choose (often surrogate) outcome measures for establishing 'efficacy.'"; in another, that the quality checklists and risk of bias tools of evidence-based medicine "may be unable to detect the increasingly subtle biases in industry sponsored studies" (Greenhalgh et al. 2014). Who chooses the endpoints, and what risk of bias tools can detect, are two things. What needs checking here is whether a rating of low risk of bias has been used to end a challenge to the choice of endpoints. Risk of bias tools do not review "whether this endpoint ought to be the endpoint"; if their conclusion is taken to answer that question, it is misrouting. No instance of closure was found.
The statistical significance threshold. Paper 2 said that where this threshold sits has been questioned all along (Wasserstein & Lazar 2016; Benjamin et al. 2018). A possible misrouting: a paper's statistical methods pass peer review, with the conclusion "the analysis is correct," and this conclusion is cited as having answered "judging this kind of decision by 0.05 is appropriate." The first is judging by p; the second is questioning p. No instance was found.
A site to be tested: procedural appeals. The Board of Standards Review of the American National Standards Institute reviews whether a standard has gone through due process; its rules state that it "does not evaluate the content of standards and does not hear appeals with regard to technical issues" (ANSI, description of the Board of Standards Review's responsibilities). This in itself is a clear narrowing, not misrouting. The possible misrouting comes after the conclusion is sent out: if a conclusion of procedural compliance is taken to end a challenge to a technical premise, that is misrouting. The rule text is primary; no individual case was found.
To be excluded: GRADE. Some would take GRADE, the system for grading evidence, as an example of misrouting: it rates the certainty of evidence, and its conclusion seems to be taken to answer "should this be done." This paper does not use it. Before rating evidence, GRADE first rates which outcomes count and how important each is (GRADE Handbook); it places a line of the normative ledger at the start of the procedure. How it handles this line, who does the rating, and whether those affected can get in, are the business of Paper 4.
Pushing back falls at the step where an individual case is closed: rule texts can show the grounds for closing a case and their limits; whether a particular case was pushed back depends on the actual record of what was cited and how it was handled (Paper 1, §9). That none was found is evidence neither that misrouting is rare nor that it is common.
Where This Paper Stops
This paper does not prove that evidence review is wrong; it does not prove that the ought in any particular standard is wrong. Finding misrouting only compares scopes and does not depend on the claim that "normativity is not conferred by experience": a reader who does not accept that claim can still use this paper's six fields.
What it hands to Paper 4: whether misrouting can last depends on where the function of questioning the ought lies, and how large it is. Paper 4 looks at seats.
The remainder it leaves: no individual case of pushing back was found in the public records examined; the pathway in §6 has no instance in the rules examined; the site of risk of bias and surrogate endpoints can be placed only once an instance of closure is found.
Notes
- The public records this paper examined fall into the following kinds: NICE stakeholder comments tables for guideline updates; the appeals rules of the Board of Standards Review of the American National Standards Institute and of the IEEE Standards Association; the GRADE Handbook; Commission Implementing Decision (EU) 2017/1996; see also Paper 1, Note 1.
The Six Propositions of This Paper
- S15 Three scopes. What it is authorized to handle; what it actually handled, and what its conclusion can prove; what consequences it is given. Registered separately; judged by the text of authorization and the record of handling, not by name. (Conceptual distinction; on the same road as Douglas 2000)
- S16 Misrouting and its two directions. Misrouting is one kind of transfer of use: a challenge to p is registered as answered by the conclusion of a review that did not handle it or whose conclusion's scope falls short of what it is said to have answered; scopes compared by text; the three shapes are not mutually exclusive, and cases are registered by six fields; sending forward and pushing back address different parties; an answer that reached the matter with retention is not judged misrouting. (Conceptual distinction: definition; six fields: notation)
- S17 Transfer of use beyond the scope of proof. Passing a review proves the part actually handled; the end of a procedure is not a substantive answer, and a limited scope does not prove that anyone claimed more than was theirs; consequences beyond the scope of proof are transfer of use, and transfer of use that goes as far as registering as answered is misrouting; action beyond authority checked separately. (Descriptive; same form as Institution Theory Z12, with the "line" of the original replaced by a line on the normative ledger, own undertaking)
- S18 Neither failure of duty nor dissembling. Misrouting can happen in the course of good-faith duty; hiding a position behind science is a different mechanism, checked separately. (Conceptual distinction)
- S19 An expanded mandate cannot, by itself, supply it. Keeping the membership condition "may not deny p" does not supply independent questioning; the possibility of internal rewriting does not prove that an addressable arrangement exists; checked separately. (Conceptual distinction; by Paper 2, S12; takes over Institution Theory Z34)
- S20 A pathway to be tested. Accepting only the conclusions of evidence review as grounds for closure, while excluding normative questioning from other effective procedures; one possible pathway, not the common source; checked from actual rules; compatible with S18. (Descriptive: mechanism to be tested)
References
I · This system
- Qin, H. (2026). Not Crossing the Line (SAE Institution Theory, Paper 2) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23270384
- Qin, H. (2026). Open to Questioning (SAE Institution Theory, Paper 6) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23271324
- Qin, H. (2026). Parochialism (SAE Theory of Disciplines, Paper 1) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.24177748
- Qin, H. (2026). Standard-Setting (SAE Theory of Disciplines, Paper 2) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.24177754
II · Rule texts and institutional documents
- American National Standards Institute. Description of the responsibilities of the Board of Standards Review (current version). ansi.org.
- Commission Implementing Decision (EU) 2017/1996. Official Journal of the European Union.
- National Research Council. (1983). Risk Assessment in the Federal Government: Managing the Process. National Academy Press.
- Schünemann, H., Brożek, J., Guyatt, G., & Oxman, A. (Eds.). (2013). GRADE Handbook.
- Wasserstein, R. L., & Lazar, N. A. (2016). The ASA statement on p-values: Context, process, and purpose. The American Statistician, 70(2), 129–133.
III · External literature
- Benjamin, D. J., et al. (2018). Redefine statistical significance. Nature Human Behaviour, 2, 6–10.
- Douglas, H. (2000). Inductive risk and values in science. Philosophy of Science, 67(4), 559–579.
- Greenhalgh, T., Howick, J., & Maskrey, N. (2014). Evidence based medicine: A movement in crisis? BMJ, 348, g3725.
- Hempel, C. G. (1965). Science and human values. In Aspects of Scientific Explanation (pp. 81–96). Free Press.
- Longino, H. E. (1990). Science as Social Knowledge: Values and Objectivity in Scientific Inquiry. Princeton University Press.
- Pielke, R. A., Jr. (2007). The Honest Broker: Making Sense of Science in Policy and Politics. Cambridge University Press.
- Weinberg, A. M. (1972). Science and trans-science. Minerva, 10(2), 209–222.
Interfaces
Main upstream: Institution Theory Z12, Z34; Paper 1, S3, step 4; Paper 2, S12.
Also used: Weinberg 1972; the Red Book 1983; Douglas 2000; Hempel 1965; Pielke 2007; Greenhalgh et al. 2014; the rules of ANSI's Board of Standards Review; Longino 1990 (in the text, §5).
This series: S15 to S20.
Handed to Paper 4: where the function of questioning the ought lies, and how large it is.