Wikipedia talk:No original research/Archive 41

Archive 35

←

Archive 39

Archive 40

Archive 41

Archive 42

Archive 43

→

Archive 45

Question

If a primary source is summarized in so that the only two views on the source held by any editors are 1) The source says X, and 2) I do not understand the source sufficiently to verify it, do we have a problem? That is, is the "specialist knowledge" clause actually an issue if there are not two competing viewpoints of it - i.e. if no editors actually disagree about what the source says?

I'm wondering if this simple switch - requiring not the agreement of all editors, but the lack of disagreement of any editors - is not sufficient to solve the issue here. Phil Sandifer (talk) 04:14, 4 January 2009 (UTC)

I am not sure what you are asking here Phil... If you are asking whether can a specialist introduce his/her own OR, as long as no one challenges it... the answer is no. If you are asking something else, then please expand your question so we can examine it. Blueboar (talk) 16:04, 4 January 2009 (UTC)

Well, that becomes a bit of a "tree falls in a forest" situation. If nobody challenges it, in practice, yes, a specialist can introduce his own OR, or his own vandalism, or anything else. I mean, that's beside the point. What I'm asking is whether "I don't understand the source, so I can't verify the claim" is a valid challenge or not.

I suspect that it's not. If you think it is, though, I invite you to go find a thoroughly-sourced article on a specialist topic yo don't understand and remove all citations to sources you don't understand well enough to verify. I'm guessing you'll be reverted. Phil Sandifer (talk) 16:24, 4 January 2009 (UTC)

I would agree that "I don't understand the source" is not a valid challenge. This is why we removed the word "easily" from the passage (which, as you know, used to say things had to be "easily verifiable"). Sometimes verifying a statement isn't going to easy.

That said, I would add a caution to what I just said... responding to a challenge by saying "You don't understand the source" is not a valid response to a challenge. Blueboar (talk) 16:47, 4 January 2009 (UTC)

That's fair. I'm wondering if the issue is that we're using "specialist knowledge" when what we mean is more accurately a matter of credentialism. I mean, it seems like we're both OK with the idea that maybe someone who doesn't understand papers on a scientific topic is not the correct authority for whether a summary of one of those papers is accurate. What we're not OK with is a "MS in chemistry or don't edit" policy. Perhaps a phrase like "specialist qualifications" or something would be better - focusing it on the disinterest in the degree, as opposed to the knowledge. Phil Sandifer (talk) 17:04, 4 January 2009 (UTC)

I understand what you are saying, but I think your use of the word "authority" is off. No one has more "authority" than anyone else. Yes, we value specialists (and specialists do not necessarily have a degree), because specialists will have more knowledge of the subject. But that does not translate into having more authority on Wikipedia. If anyone, including a non-specialist, challenges what a specialist writes as being OR ... the specialist should go to the talk page and discuss the problem. He/she needs to convince the other editors that what he/she wrote is not OR. That is how the system works. Blueboar (talk) 17:52, 4 January 2009 (UTC)

Yes. But the established way of doing that is citing sources. Now, if the specialist cites sources that the other editors don't think support the claims, that's one thing. But I'm very much skeptical on whether we need to carve out any sort of special protected status for people who do not understand the source being cited one way or another. And it seems to me that the current language does so. At this point, I see two small changes that I think adequately handle the situation - change "can be verified" to "would not be disputed," and "specialist knowledge" to "specialist qualifications." This, I think, moves the burden of dispute to disagreement instead of mere puzzlement, and clarifies that the issue is about credentialism. Thoughts? Phil Sandifer (talk) 18:09, 4 January 2009 (UTC)

These scenarios keep taking us into cases with exaggerated extremes, cases that should send up a half dozen red flags besides this quibble about "specialist knowledge". Why is it that WP could possibly be well served to pave the way for anonymous editors who self-proclaim themselves "specialists" to contribute basically unverifiable interpretations of primary source documents? How can notability be judged on a subject that secondary sources have completely ignored? How can we defer to the say-so of editors who self-report they have specialist knowledge of the topic but who haven't been vetted? How can NPOV be judged-can't those with specialist knowledge have biases? How can consensus be formed without an advantage given to those who say, "just trust me-I know what it means"? How can readers trust the contents of the articles? How can fringe views be kept in proper perspective? How would this not lead to an infinity of idiosyncrasy and esoterica littering the encyclopedia? How would this not open the doors and windows wide to let in all manner of undetected original research misrepresented as too advanced for others to judge? Professor marginalia (talk) 18:18, 4 January 2009 (UTC)

For the most part, those seem like non-sensical questions that have little to do with this proposal. However:

• Why is it that WP could possibly be well served to pave the way for anonymous editors who self-proclaim themselves "specialists" to contribute basically unverifiable interpretations of primary source documents?
  • We're not talking about "basically unverifiable interpretations" here. The question is really whether an attempt to verify something that results in "Hm, I can't tell" is a red flag (As opposed to "It doesn't appear to me to say that," or "it clearly does not say that") - particularly when we are dealing with specialist topics.
• How can notability be judged on a subject that secondary sources have completely ignored?
  • Not at issue - nobody, least of all me, is suggesting the abandoning of secondary sources. This deals with the use of one type of source - it doesn't alter the necessity of other types.
• How can we defer to the say-so of editors who self-report they have specialist knowledge of the topic but who haven't been vetted?
  • Well, what's the more appropriate course of action for an editor on a topic who does not understand key sources for that topic? I would suggest that if I were to start editing articles in an area where I did not understand the sources, telling me to go away and let people who understand the topic work would not be an inappropriate response. Again, we're distinguishing here between two things - "I don't understand the source, so it can't be used that way" is not a valid argument. "Looking at the source, I do not think it says that" is a valid argument, and one that cannot (and should not) be refuted by "But I'm an expert." And that's the key distinction - right now, any editor's lack of understanding is an argument for removal. That bar is ridiculous. The bar should be "any editor's disagreement," which is a higher bar. And it should remain the case that claimed expertise is not a refutation in the case of a disagreement.
• How can NPOV be judged-can't those with specialist knowledge have biases?
  • And? Those without specialist knowledge can have biases. Everyone can have biases. The expectation is that they will try to work through that.
• How can consensus be formed without an advantage given to those who say, "just trust me-I know what it means"?
  • What value is a consensus of people who openly don't understand the relevant sources?
• How can readers trust the contents of the articles?
  • By following up on the cited sources, same as any other article.
• How can fringe views be kept in proper perspective?
  • By still restricting ourselves to reliable sources. Which this doesn't undermine.
• How would this not lead to an infinity of idiosyncrasy and esoterica littering the encyclopedia?
  • How would it lead to this infinity? We're still talking about summarizing reliable sources.
• How would this not open the doors and windows wide to let in all manner of undetected original research misrepresented as too advanced for others to judge?
  • Again, this doesn't seem to me to be at issue here - if someone reading the source disagrees with the summary, the summary is either inaccurate or OR. The issue here is the case where someone reading the source neither agrees nor disagrees with the summary. In the current wording of the policy, such a situation is taken as evidence of OR. That doesn't make sense.

Again, the key thing here is that "I don't understand the source" should not be a valid grounds for any alteration of the source's use, in any circumstances. The minimum threshold for an OR dispute should be "I do not think that the source clearly says that," not "I do not know if the source clearly says that." And the acceptable grounds for argument after that are not references to credentials, but close, careful reading of the source and attempts to work through the disagreement and find a mutual consensus.

To go with a hypothetical here, and, reluctantly, to return to Derrida, since that's the book I have handy to frame this argument, let's say that an article made the claim that "Derrida criticizes Searle for asserting that Derrida deals with the permanence of written texts, pointing out that his concern with permanence extends to other forms of communication." This claim is cited to Derrida's book Limited Inc, specifically to page 52.

Let us imagine two editors who object to this claim as OR.

Editor 1 says: "Looking at page 52 of Limited Inc, I do not understand what is going on there - there's a lot of specialist terms, so this is not verifiable to me. It might well say that. It might not. I can't tell."

Editor 2 says: "Looking at page 52 of Limited Inc, it appears to me that Derrida is not arguing with Searle, but a guy named Sarl, and that he actually says that his concern with permanence extends to other graphemic marks, not other modes of communication.

Editor 1's objection is a non-starter. We do not want to allow objections such as this to carry any real weight.

Editor 2, on the other hand, raises two visible complaints. Now, as it happens, to anyone familiar with Derrida and with Limited Inc, these are very silly complaints. However it is not appropriate to just say "Don't be silly, you clearly don't understand the book, trust me, I'm an expert." Rather, the argument in response to Editor 2 is "Actually, if you go earlier in the book, you see that Derrida elects to refer to Searle as Sarl throughout the essay as a sort of mockery. And a grapheme actually just refers to a communicative mark - that is, some marking of any sort that is meant to communicate." And this could then move, as needed, to the finding of sources that back up both of those claims (an earlier page reference to Limited Inc.

The goal should be to allow objections of the form of Editor 2's, allow responses of the form of explaining what "Sarl" and "grapheme" mean here, but to forbid dismissal of those objections by just saying "I'm an expert," and to forbid objections of the form of Editor 1's. I will support any proposal that clearly accomplishes those three goals. As it stands, however, Editor 1's objections are allowed, and the response explaining what "Sarl" and "grapheme" mean is unclearly allowed. (All of this, of course, is prefaced by the statement "assuming we are dealing with an article where Limited Inc. is a primary source") Phil Sandifer (talk) 18:53, 4 January 2009 (UTC)

Why isn't it that "the goal should be" to call upon secondary sources? The debate between Searle and Derrida is infamous and widely written about. books articles Professor marginalia (talk) 19:39, 4 January 2009 (UTC)

Oh for God's sake. You're just playing silly buggers now. We've gone over this issue how many times now? I used Derrida here because the book was five feet from where I was sitting, unlike any other specialist sources I could quickly find a good example in. I know secondary sources exist for it. However, it still works to demonstrate issues of summary. Phil Sandifer (talk) 19:56, 4 January 2009 (UTC)

Really? I'm not alone asking, and still waiting, for a solid, realistic example where this clause creates insurmountable problems. We prefer secondary sources in such cases as this. Rather than acting as an impediment, the clause in this dispute stimulates editors to uncover superior sources for the claim. How is this example, then, one of a problem with policy rather than a benefit? Professor marginalia (talk) 20:06, 4 January 2009 (UTC)

I've indicated the existence of dozens of debates in Critical Inquiry. Which, as I said, was one journal - plenty of other debates exist for subjects that pass WP:N but don't have the depth of coverage in secondary sources of the Derrida/Searle debate. Phil Sandifer (talk) 22:52, 4 January 2009 (UTC)

An example taken from a dispute in the encyclopedia, PS. That requires a cite to a primary source text that received no notice in secondary sources, and is unverifiable except to those with "specialist knowledge". Professor marginalia (talk) 04:01, 5 January 2009 (UTC)

Reality check time... hypotheticals are fine.... but how likely is it that anyone is ever going to raise an OR objecting on the grounds that "I don't understand what the source is saying"? I really don't think this will ever happen. I think people only raise OR objections in two situations: a) when information is not cited, or 2) they think a cited source does not back the information in the article. In the latter case, the objecting editor inevitably thinks that he/she does understand what the source says. What they disagree with is what the article says. Can anyone point to a real life situation where someone complained that they didn't understand the source? Blueboar (talk) 19:47, 4 January 2009 (UTC)

I have found that closing overly literalist and rules-lawyery loopholes - particularly when those loopholes are all but explicitly called for in the policy - which this still is. I have seen very few cases where overly rigorous and restrictive language has not gone on to be abused. Phil Sandifer (talk) 19:56, 4 January 2009 (UTC)

A general-purpose online encyclopedia already exists with policies that might better achieve some of the objectives Phil Sandifer appears to seek in his comments above. It can be found at http://www.citizendium.org. However, note that among Citizendium's policies also is-- you guessed it: "no original research". Note how the Citizendium policy too is written in such a way as to indicate a general avoidance of primary-source material but without necessarily prohibiting it. The questions Phil Sandifer has raised at extreme length in the past month or so (getting close to a megabyte now, which is a very large amount of text) appear to me to have little to do with writing an encyclopedia, be it Wikipedia or any other encyclopedia. They're much more like questions about how to do original research such as academics do, in a wiki that as a matter of policy forbids original research. Sorry to be quite this direct, but it seems to me that's about what it boils down to. ... Kenosis (talk) 19:59, 4 January 2009 (UTC)

Citizendium is a piece of shit project with idiotic rules on expertise that won't work. Wikipedia's rules on credentialism are vastly more serious and sane. It betrays a pathetic desire not to seriously engage issues to simply cry "Go to Citizendium" every time some issue remotely related to specialization comes up. Now, nothing I have said has been about de-emphasizing secondary sources. It has, however, been about adopting sensible policies about how to use primary sources in the cases that they are necessary - cases that we (along with CZ) recognize the importance of. As for your suggestion that this is about "how to do original research such as academics do, in a wiki that as a matter of policy forbids original research," with all due respect, this is stark idiocy. First of all, the line between the sort of description of Derrida I use in the example just above and academic research is not so much a line as an ocean. There is not even a basis for comparison of the sort of basic summary of difficult sources I'm talking about here and academic research. Second of all, were I engaging in academic original research, I assure you, I would be looking for a better place than Wikipedia to do it. I apologize if the prospect of actually having to read a megabyte of discussion of an issue is onerous for you, but if the level of your reading is going to be this worthlessly shallow, frankly, we're both better off if you don't bother. Comments such as this, which manage the impressive feat of being both dismissive and completely off-base, are worse than useless. Phil Sandifer (talk) 20:08, 4 January 2009 (UTC)

I didn't say your extensive and varied arguments at-length are "onerous" to me. I said, roughly, you've got it backwards. This is an encyclopedia where the rules are, essentially, to write in summary style, as is also the case in Citizendium. What you're arguing towards here and in the nearly megabyte worth of text over the past month is essentially to do the opposite, to make the policy allow you to go digging into all the primary sources that already have secondary sources associated with them, neglecting those ample secondary sources in preference of your own research of the primary sources. Sorry, but within Wikipedia, that's called... "original research". ... Kenosis (talk) 20:25, 4 January 2009 (UTC)

You are so completely disengaged from anything I have said or come close to saying that it is laughable. I have not argued for neglect of secondary sources, I have not argued for "digging" of any sort. I have argued for the use of summary style. Your dominance over the straw man you have created is unquestioned. Now, kindly, if you're not going to answer what people are saying, go away. Phil Sandifer (talk) 20:30, 4 January 2009 (UTC)

Phil, the impression others get from your various arguments is that you want the policy to say "People should be alowed to include their own Original Research if they have specialist knowledge". That may not have been your intent, but that is how others see it. Blueboar (talk) 22:46, 4 January 2009 (UTC)

I am unable, rereading my comments, to see how anyone could think that I am saying that except through willful density. To call descriptions of a source that are uncontroversial to anybody who understands the source, and are undisputed "original research" is a bizarre perversion of that term. Phil Sandifer (talk) 22:52, 4 January 2009 (UTC)

Um... the policy allows for descriptions of a source that are uncontroversial and undisputed. In fact, whole point of the section is to say that such descriptive comments are not OR. Blueboar (talk) 23:06, 4 January 2009 (UTC)

That's wonderful news. My proposed addition in the bottommost section then, I assume, is uncontroversial? :) Phil Sandifer (talk) 00:00, 5 January 2009 (UTC)

Pointing out what an author doesn't say

I'm having a discussion with someone about statements such as " However Dolphin does not offer any explanation for why the same type of erosion patterns are not present to the same extent on the many other rock surfaces at Giza." which I've removed [1] - is this actually explicitly covered anywhere? It seems pretty obviously OR, but the editor disagrees. I don't see how you can do this without it being OR, but others may disagree. dougweller (talk) 14:26, 5 January 2009 (UTC)

It is OR, because the source does not directly support the information as presented. Honestly, sometimes I think this project is doomed, given that people just enjoy writing about their own ideas more than they enjoy summarising what reliable sources have written. Cheers, Jayen466 14:45, 5 January 2009 (UTC)

Agreed... what is needed is an attributed source for the comment as to what the Author does not say. Something like: "However, as (reliable source X) notes, Dolphin does not offer any explanation..." Blueboar (talk) 16:59, 5 January 2009 (UTC)

Lost in the wash

Without a secondary source, a primary source may be used only to make descriptive claims, the accuracy of which is easily verifiable by any reasonable, educated person without specialist knowledge.

The current wording is a change from the agreed wording earlier this year. As a delve into the archives will show, there was a misunderstanding over "only to make descriptive claims", some took that to mean that primary sources could not present interpretive, or explanatory information. (see the archives for Wellington's analysis of the Battle of Waterloo "the nearest-run thing you ever saw in your life." ) it was agreed at that time that primary sources can be used to make such analysis hence the wording:

Wikipedia articles should rely on reliable, published secondary sources. All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source, rather than original analysis of the primary-source material by Wikipedia editors.

Primary sources that have been published by a reliable source may be used in Wikipedia, but only with care, because it is easy to misuse them. For that reason, anyone—without specialist knowledge—who reads the primary source should be able to verify that the Wikipedia passage agrees with the primary source. Any interpretation of primary source material requires a reliable secondary source for that interpretation. To the extent that part of an article relies on a primary source, it should:

• only make descriptive claims about the information found in the primary source, the accuracy and applicability of which is easily verifiable by any reasonable, educated person without specialist knowledge, and
• make no analytic, synthetic, interpretive, explanatory, or evaluative claims about the information found in the primary source.

I think that the current wording needs changing to make sure that this misunderstanding of descriptive claims is not made. --PBS (talk) 18:22, 31 December 2008 (UTC)

If you are saying that we can report that a primary source contains an analysis (such as Wellington's analysis of Waterloo), and mention what that analysis is... I absolutely agree. That clearly isn't Original Research. That is a simple descriptive statement about something Wellington said. But I don't see how the current language is any different in this reguard. Blueboar (talk) 18:44, 31 December 2008 (UTC)

You an I both, but that was not the interpretation of User:COGDEN back at 00:26 on 20 November 2007, see Wikipedia talk:No original research/Archive 28 and my comment at 18:13 on 28 November 2007 Wikipedia talk:No original research/Archive 29 The discussion was resolved with User:Kenosis making this edit which altered "make no analytic, synthetic, interpretive, explanatory, or evaluative claims, unless such claims are verifiable from another source." to "make no analytic, synthetic, interpretive, explanatory, or evaluative claims about the information found the primary source, unless such claims are verifiable from another source."

What we mean is not that "a primary source may be used only to make descriptive claims" but "Wikpedia editors can only include accurate descriptions of the content of primary sources that is easily verifiable ..." (or something along those lines because Wellington's comment can interpreted as a expert analysis not a descriptive claim which would certainly cover Wellington's statement "at about ten o'clock he commenced [an] attack upon our post at Hougoumont") --PBS (talk) 22:40, 31 December 2008 (UTC)

Well... I suppose someone could misinterpret the current intent if they take that sentence out of context. I have no objections to your clarification. Blueboar (talk) 14:39, 1 January 2009 (UTC)

OK, based on PBS's concern here, I have returned some of the older language... while keeping the current format. Feel free to revert if I miss the mark. Blueboar (talk) 14:23, 2 January 2009 (UTC)

rerverting, messy, wording is a bit muddled:

• Without a secondary source, editors should avoid making analytic, synthetic, interpretive, explanatory, or evaluative claims about the information found in a primary source
• the fact is that editors should not make analytic, synthetic, interpretive, explanatory, or evaluative claims about any information, no matter what sources they have. PBS's version above reads very differently. Semitransgenic (talk) 14:38, 2 January 2009 (UTC)

On 2nd Blue Bore made this edit which was revered by Semitransgenic. Semitransgenic please see Wikipedia_talk:Reliable_sources#Primary_sources given that there is confusion, what you propose as improved wording for this section? --PBS (talk) 23:07, 6 January 2009 (UTC)

Getting me to go away

Since it seems to be on so many people's minds lately...

Here is one change - one change - that would satisfy me on the specialist knowledge issue, on the interpretation/description issue, on every issue I have with this policy, get me to remove the discussion tag, and leave this page in peace.

Make clear that descriptions of sources that nobody disagrees with are OK. Period. If nobody actually thinks a claim about a source is not reflected in the source, it is not OR.

This moves the test away from being some hypothetical educated person with poorly defined attributes, and stops the problematic gap whereby anybody who even fails to understand a source has effective veto power over the source's use (after all, *any* reasonable, educated person has to be able to verify it).

That's it. Don't remove sourced material unless you actually think the source does not clearly support it. It can be done in one sentence, and serves as an adequate guard against the problems that the language of this policy has. Phil Sandifer (talk) 23:53, 4 January 2009 (UTC)

I assume you wish to keep the idea that interpretative statements about primary sources would not be OK.Blueboar (talk) 00:02, 5 January 2009 (UTC)

I still don't like the interpretation/description divide, but I think that the explicit safeguard of "If you don't actually think it's unclear in the source, don't remove it" is sufficient to hedge against the worst problems of that divide. Phil Sandifer (talk) 00:06, 5 January 2009 (UTC)

See section below. I like where you're coming from, Phil, but I still think there are better ways of addressing your concerns (about such things as representing a beleaguered living person's defence against a multitude of critics cited in their BLP) than using primary sources. Even so, as others have pointed out, policy actually allows the "careful" use of primary sources. And that already causes all sorts of mischief. Cheers, Jayen466 01:28, 5 January 2009 (UTC)

I am guardedly OK with this direction... I may have second thoughts when I see it in the policy (I would need to see the exact wording and how it interacts with the wording in the rest of the secton. Context is everything). —Preceding unsigned comment added by Blueboar (talk • contribs) 03:35, 5 January 2009

I want to see the wording also. For example, the caveat "unless you actually think the source does not clearly support it" isn't useful with texts that are vague or ambiguous. We have to be careful while closing the gaps on for any and all potential or hypothetical problems not to overlook the obvious and present ones. Professor marginalia (talk) 04:21, 5 January 2009 (UTC)

Well, I would say that vagueness and ambiguity can also be described. But I'll work on wording - I'm travelling all day tomorrow, so I'll probably have something on Tuesday. Phil Sandifer (talk) 04:34, 5 January 2009 (UTC)

OK, but what about the cases that are not vague or ambiguous, just opaque? By opaque, I mean texts that take more than a passing interest to read and describe, possibly requiring reading other texts, so they are not suitable for drive-by editing. I don't see any problem with requiring specialist source based research--that's not original research. This is the problem I have with the specialist "knowledge" wording. I think it would be better replaced by specialized "training". This would make it clear that doing source-based research to collect specialist knowledge is OK (in fact it's essential), but expert training is not needed. Dhaluza (talk) 12:35, 6 January 2009 (UTC)

Dhaluza... Please read the Policy again... the "specialist knowledge" clause is not talking about the editor doing research and whether he/she has specialist knowledge. It is talking about the reader who wants to verify what the article says and whether that reader must have specialist knowledge.

In the hypothetical situations Phil keeps raising, someone (possibly a specialist, possibly not) has already done the source based research. Someone has already read the source and has written a summary of it in a Wikipedia article. Phil's problem is about what comes next... verification. He contends that there are some primary sources that are so complex and difficult to understand that it is impossible for anyone who does not have "specialist knowledge" to understand it. He contends that, while a specialist may be able to write a summary of such primary sources, it would be impossible for that summary to be verified by a non-specialist... because a non-specialist could never understand the primary source the summary was based upon. Blueboar (talk) 15:16, 6 January 2009 (UTC)

Well, or more troublingly, because our policy is not written with the threshold that at least one non-specialist could understand the source but rather that all non-specialists could understand the source, I'm concerned that difficulty in understanding becomes a barrier to inclusion very, very quickly. Phil Sandifer (talk) 15:39, 6 January 2009 (UTC)

BLINK... Hold on a sec... can this all be resovled by a simple change of an article? From:

• "....the accuracy of which is verifiable by any reasonable, educated person without specialist knowledge."

to:

• "... the accuracy of which is verifiable by a reasonable, educated person without specialist knowledge."

Would this resolve the issue? Blueboar (talk) 16:03, 6 January 2009 (UTC)

Either way. ... Kenosis (talk) 16:22, 6 January 2009 (UTC)

That also strikes me as a good change for that issue - it removes the ridiculous burden of proof that is shifted to the side that is already doing source-based research. I still think that a change along the lines of "Don't challenge what you don't actually think is unsupported" is appropriate. Phil Sandifer (talk) 16:24, 6 January 2009 (UTC)

I think you have a good point there when you note that there exists a "side that is already doing source-based research". Presently the wiki is very much in need of more persons doing research and sourcing, and perhaps a few less drive-by taggers. I've seen many productive contributors driven away, apparently frustrated by knee-jerk critics who seem to like to suddenly show up only when someone is doing the real work. But I'm digressing somewhat. ... Kenosis (talk) 16:40, 6 January 2009 (UTC)

Thank God! one issue resolved (change in article has been made).

Now, as to your "don't challenge what you don't actually think is unsupported" proposal ... I think this is an unnecessary instruction. Seriously, is this really a problem? Does anyone ever challenge a statement that they think is supported by the source? Has this ever happened? Blueboar (talk) 16:45, 6 January 2009 (UTC)

Well, my bigger concern is challenging stuff they don't know whether is supported by the source - but the larger issue here is that this ties into the description/interpretation divide (which, when last we left off, nobody seemed to quite agree on the meaning of). This, to my mind, is important because it helps focus disputes about what a source says *on the source*. I have seen vague claims that "it could be unclear" used to remove stuff that it does not seem to me anyone would actually argue themselves is unclear. And that is where I think this is important - because it requires actually looking at sources, discussing them, making sure we're summarizing them correctly, etc. What I'm looking for is language that stresses that objections and challenges are source-based too. Phil Sandifer (talk) 18:36, 6 January 2009 (UTC)

I think you are essentially asking "Burden of evidence" questions... if so... I have started a thread on this issue below. Blueboar (talk) 18:48, 6 January 2009 (UTC)

"add no new information to what is already present"

Not working for me. How about "add no information to what is already present" or "add no new information" or "introduce no additional information"? - Dan Dank55 (send/receive) 00:33, 6 January 2009 (UTC)

Does this work? ... Kenosis (talk) 00:48, 6 January 2009 (UTC)

Either "new" and "what is not already present" mean the same thing (in which case, delete one as redundant), or else "new" means something else (in which case, we should say what that is). - Dan Dank55 (send/receive) 13:03, 6 January 2009 (UTC)

It's phrased ok--"add no new information to what is already present." That is, saying 30 is 50% of 60 is "new" in a sense but was already implicit when you have 30 and 60 in the source. Rjensen (talk) 14:30, 6 January 2009 (UTC)

Propose keeping a record of decisions for WP:NOR

Laws, rules, or guidelines can be subject to different interpretations. That's why organizations from the U.S. legal system to the U.S. Golf Association (USGA) keep a record of clarifying decisions that were made, based on the laws or rules. For the U.S. legal system it's called case law, and for the USGA it's called decisions. The USGA Rules book not only has the rules but also clarifying decisions that were made for actual situations that occurred on golf courses. These are equivalent to examples that people can look up to see how to better interpret the rules. USGA might be a good example for us. Perhaps a similar system for WP:NOR would help if someone needs an interpretation of WP:NOR, such as the request made in a previous section, "Failing the 'directly related' test - Examples please" . --Bob K31416 (talk) 05:34, 4 January 2009 (UTC)

See also WP:NOTLAW, but the archives of WP:NORN won't do? WhatamIdoing (talk) 07:28, 4 January 2009 (UTC)

Too legalistic... See Wikipedia:The rules are principles. Blueboar (talk) 15:56, 4 January 2009 (UTC)

This wasn't meant to be legalistic but to be a source of examples that would enhance and clarify the guidelines.

As it stands now, it would be very difficult for an editor to search WP:NORN to find the case that applies to that editor's case. The current proposal is to use a small percentage of WP:NORN cases for examples that are relatively independent of each other, i.e. not redundant. The cases would be simplified and summarized and easy to read for guidance. As I mentioned, they would be used to help people with questions like the one regarding "directly related". --Bob K31416 (talk) 17:30, 4 January 2009 (UTC)

Admittedly this approach would have the potential benefit of making a summary of past discussions available for quick review. On the down side, it would require either (1) something akin to an omniscient third party to monitor and sum up the prior discussions, or (2) an additional layer of consensus-based discussion about how to sum up the discussions for future use. I don't think it's feasible. ... Kenosis (talk) 18:10, 4 January 2009 (UTC)

I support this proposal. --Phenylalanine (talk) 18:31, 4 January 2009 (UTC)

You're hired. Start here. ;-) . ... Kenosis (talk) 18:48, 4 January 2009 (UTC)

I was thinking of something a little more noticeable. For example, a separate page titled WP:No original research/examples (WP:NORE). At the end of each section of WP:NOR there would be a link to the corresponding section of examples at WP:NORE. --Bob K31416 (talk) 20:44, 4 January 2009 (UTC)

Phenylalanine, Thank you for your support.

Kenosis, Thank you for noticing that benefit, although it wasn't intended as a summary of past discussions but a short list of good examples to help clarify WP:NOR. Regarding your items 1 and 2, the page could be constructed and maintained the same way as any page on Wikipedia. I think that it would help to reduce conflict and excessive discussions by being informative. --Bob K31416 (talk) 20:58, 4 January 2009 (UTC)

WP:NPOV has a rough equivalent to what you appear to be proposing, if I'm reading your clarification correctly. It's at WP:NPOV/FAQ. Is this the sort of thing you have in mind? If so, I should say there are some very experienced editors, intimately familiar with practical application of this policy, who weigh in on this talk page to varying extents and in varying combinations, some of whom might perhaps be interested in further discussing whether to develop a sub-page like this. Any interest? comments? criticisms? ... Kenosis (talk) 22:14, 4 January 2009 (UTC)
..... I should add a note of caution that there've been some arguments on the WT:NPOV talk page about how the FAQ page came to be labeled as "policy", but without being maintained with the same degree of attention as the policy page itself. ... Kenosis (talk) 22:38, 4 January 2009 (UTC)

It would be different than the FAQ situation since it is taking examples from WP:NORN. An editor seeking clarification on some aspect of WP:NOR could see how an issue related to that aspect was treated before. --Bob K31416 (talk) 00:21, 5 January 2009 (UTC)

This makes sense to me, if somebody has the time and the inclination to do it. There are plenty of good examples to pick from at WP:NORN. ... Kenosis (talk) 17:14, 5 January 2009 (UTC)

Re "This makes sense to me, if somebody has the time and the inclination to do it." - Thank you for your support. Once it gets started, I expect that it will be built and improved by the contributions of various editors, similar to the way an article page is developed. The primary difference is that the only source for the contributions will be WP:NORN. Here's how it might get started:

1. Create the page WP:No original research/examples (WP:NORE)
2. At the top of the page, give instructions for contributing to the page. Here's some preliminary thoughts on what the instructions might include:

• • Place the contribution in the section in WP:NORN that has the same title as the relevant section of WP:NOR.
  • Use only WP:NORN as a source. Do not express your opinion or ideas.
  • Cite the section of WP:NORN that is the source.
  • Do not simply copy the discussion from WP:NORN but summarize succinctly.

Once it gets started, it should evolve and improve like any Wikipedia page, with the efforts of various editors who choose to contribute. The first step is to get the OK to start the page. --Bob K31416 (talk) 00:28, 6 January 2009 (UTC)

By all means, feel free to create the page, maybe put an {{essay}} tag on it, and see where it goes. Just in case you were not already familiar, the process for establishing "guideline" or "policy" status is presently at Wikipedia:Policies and guidelines. ... Kenosis (talk) 05:30, 6 January 2009 (UTC)

Thank you for the good suggestions and information. --Bob K31416 (talk) 20:28, 6 January 2009 (UTC)

I have strong reservations: (a) the "case law" will require interpretation by those who use it and almost certainly by those who summarise it; (b) the whole thing will be a maintenance nightmare, and editors will wind up going back to WP:NOR, which would take precedence in any case; (c) it looks to me like a solution in search of a problem. Considering the effort required to create a summary of "case law", editors' time would be better spent on articles. --Philcha (talk) 10:19, 7 January 2009 (UTC)

Here's what I propose: there will be no third-party summarizing; instead, editors who comment at NORN on a given case are invited to briefly summarize their own comments at "WP:No original research/examples (WP:NORE)". Also, it would be useful to visibly identify what type of OR is involved: (1)"original synthesis", (2)"information not directly supported by the source cited", and/or (3)"source not directly related to the article topic". We could create a different section of examples for each of these types of original research. We also need to provide a link to the relevant discussion at WP:NORN. --Phenylalanine (talk) 12:30, 7 January 2009 (UTC)

<---- Here's an example of how one wikiproject does it in relatively NPOV way; of course, it's out of date (mea culpa) and keeping it up to day is one of the issues Wikipedia:WikiProject Israel Palestine Collaboration/Links to reliable sources discussions. CarolMooreDC (talk) 15:04, 7 January 2009 (UTC)

A list to such discussions can work for RS... but not for NOR. This is because RS focusses on sources (a limited set) while NOR focusses on the text of our articles (which can be rewritten in thousands of different ways). It is fairly easy to compile a list of discussions about whether a limited set of sources that are, or are not, considered reliable in a given topic. For any topic the same sources will come up over and over.

It is nearly impossible, on the other hand, to create a list of statements that are considered Original Research, because too much depends on context... too much depends on the exact wording of the statement, the exact wording of the source being used to support it, and the context in which these two interact. Change even one of these factors even slightly (slightly rewriting the statement for example) and you could no longer be dealing with OR. Blueboar (talk) 17:16, 7 January 2009 (UTC)

What Blueboar points out is true. But... a list of discussions will also reveal where the prose of this policy is ambiguous or otherwise problematic, and which then needs to be fixed. -- Fullstop (talk) 20:55, 7 January 2009 (UTC)

Eh... perhaps... so I guess it comes down to this: If someone wants to attempt to keep such a record, they are welcome to try. I think it too much trouble for too little reward, so don't expect me to participate, but I see no harm in others trying it. Have fun! Blueboar (talk) 21:20, 7 January 2009 (UTC)

Thank you all for your comments. The description of the proposal has evolved during this discussion. Please focus on the last form that I described and I think that would alleviate any concerns that you might have and you will see that the proposal is fairly simple and without any adverse effects. Here is an excerpt (with some copyediting and a correction) from one of my recent messages that is a fairly good description of the proposal.

Once it gets started, I expect that it will be built and improved by the contributions of various editors, similar to the way an article page is developed. The primary difference is that the only source for the contributions will be WP:NORN.

At the top of the examples page there will be instructions for contributing to the page. Here's some preliminary thoughts on what the instructions might include:

• • Place the contribution in the section in WP:NORE that has the same title as the relevant section of WP:NOR.
  • Use only WP:NORN as a source. Do not express your opinion or ideas.
  • Cite the section of WP:NORN that is the source.
  • Do not simply copy the discussion from WP:NORN but summarize succinctly.

Once it gets started, it should evolve and improve like any Wikipedia page, with the efforts of various editors who choose to contribute.

So, it is summarized discussions from WP:NORN. That's all. No new information, just information that is put in a more easily accessible form.

The editors that were involved in the original discussion at WP:NORN would be a good source of editors for contributing summaries (in line with Phenylalanine's suggestion). They would be supplemented by the efforts of other editors, in cases where useful discussions weren't summarized by any of the discussion participants. Also, the summaries that are contributed are not cast in concrete and can be edited like any other part of Wikipedia. --Bob K31416 (talk) 21:53, 7 January 2009 (UTC)

It does seem to me to have some potential to allow editors unfamiliar with the policy to get some examples of how some issues relating to WP:NOR have played out in the past. For instance, the issue with which someone is confronted might not be a WP:NOR issue, but instead a WP:V issue or a WP:RS issue-- IIRC there've been a few of those. I also think a page like this would likely give rise to arguments about interpretation. It seems to me something like this, if it's to be helpful, might need for the examples to be set in the context of how the issues were ultimately worked out in the article space and perhaps also on the article talk page. So it potentially could involve a lot more work than might at first appear necessary. But it seems possible so long as it avoids strict interpretations like the USGA example mentioned at the beginning of this thread. ... Kenosis (talk) 23:10, 7 January 2009 (UTC)

The "WP:No original research/examples (WP:NORE)" page could use (sub)section headings such as these, followed by helpful examples illustrating each type of original research (and perhaps also non-original research):

• Original synthesis
• Information not directly supported by the sources
• Source not directly related to the article topic
  • Information effectively relevant to the topic of the article (examples please???)
• Not to be confused with original research (examples)

--Phenylalanine (talk) 03:14, 8 January 2009 (UTC)

Primary sources: The novel example

Present policy contains the sentence, "For example, an article about a novel may cite passages from the novel to describe the plot, but any interpretation of those passages needs a secondary source."

In all cases where there is a substantial body of secondary literature on an author or their works, the very act of selecting a specific passage to quote in our article is OR, unless it's supported by a good secondary source that has selected the same passage for comment. We should make that explicit in the policy, otherwise we end up with articles that fail to reflect the focus and priorities of RS secondary literature.

In other words, if we are summarising the plot of a recent TV soap episode (no substantial body of secondary literature), fine, write an OR summary. But if you are dealing with a much-discussed author, do not quote the bits that seem notable to you personally: quote the bits that reliable sources discussing the author have quoted.

Any views? Jayen466 01:19, 5 January 2009 (UTC)

Good idea. This should be in a guideline not policy though. Add it to an existing guideline or write up an essay in guideline format and try to get it promoted to guideline status. davidwr/(talk)/(contribs)/(e-mail) 03:15, 5 January 2009 (UTC)

So how do you propose we word this? Blueboar (talk) 03:24, 5 January 2009 (UTC)

I would propose the following: "For example, an article about a novel may cite passages from the novel to describe the plot, but any interpretation of those passages needs a secondary source, and if there is a substantial body of secondary literature on the novel, the selection of the relevant passages should be based on and cite that literature."

Otherwise we just get a reflection of editors' idiosyncratic preoccupations (c'est magnifique, mais ce n'est pas l'encyclopédie), rather than something based on the existing body of reliable sources out there in the real world, discussing the novel, author, etc. Jayen466 10:12, 5 January 2009 (UTC)

My view: It's instruction creep, and it's confusing an overview of the plot of the work with a section on interpretation of the work. People already claim too much is WP:OR that is really a matter of WP:WEIGHT (or WP:IDONTLIKEIT, for that matter). At most, write an essay about it. Anomie⚔ 03:49, 5 January 2009 (UTC)

I think a guideline maybe. But it's not a NOR issue if the content is descriptive. I think WP should tighten up notability guidelines so that they're clearer about what kind of content is most suitable. The focus now tends to be on what kinds of subjects are notable enough for stand alone articles in the encyclopedia. Professor marginalia (talk) 04:39, 5 January 2009 (UTC)

I think that such tightening on a policy level would amount to enshrining POV pushing. There are very, very good reasons why WP:N has adamantly avoided being about article content. Phil Sandifer (talk) 04:41, 5 January 2009 (UTC)

What? Secondary sources are already one big key to determining notability. I'm not talking about specific content, as per the thesis of this section, I'm talking about what is more notable, content written about extensively in secondary sources or what's essentially a book report focused on content chosen by a wikipedian.Professor marginalia (talk) 04:52, 5 January 2009 (UTC)

Encyclopedic background information and "common knowledge" is often not the subject of news articles and other secondary intended for a popular rather than scholarly audience, and for some topics (e.g. complex scientific theories) the details may be so watered down in available secondary sources as to make them useless and we need to use the primary publication for a complete treatment of the topic. I believe this sort of thing is why WP:N specifically states that it makes no restriction on article content. Anomie⚔ 17:29, 5 January 2009 (UTC)

The WP:N guideline does give advice on content. "But the term 'notability' is still used in the sense of 'importance' to describe the level of detail that is appropriate for an encyclopedic summary. Treat each aspect with a weight appropriate to its significance to the subject. Attend to anything that may construe undue weight, including depth of detail, quantity of text, prominence of placement, and juxtaposition of statements. Keep in mind that an encyclopedia article is a summary of accepted knowledge regarding its subject, not a complete exposition of all possible details." I think we're speaking now of a situation where "a summary of accepted knowledge regarding its subject" is well represented in secondary sources but bypassed to concentrate on sharing some editor's personal "pet" ideas. This can happen with "pet" ideas found in secondary sources as well (e.g. told a funny about streaking, 'too embarrassed' to practice divorce law). Maybe this discussion is more relevant to WP:N, WP:Undue or WP:RS than W:NOR.Professor marginalia (talk) 18:18, 5 January 2009 (UTC)

"Pet" ideas expresses it quite well. You get situations where to some editor(s) it seems incredibly important to note that person X made statement Y in source Z. It usually is because the editor has a very strong opinion of the person and would like to enhance or damage their standing in the reader's eye – by quoting a statement they think is exceptionally brilliant, or by quoting one that proves the person was a cad. I would argue that if there are dozens of reliable secondary sources discussing the person, and none of them make an issue of this statement, then neither should we. Our job should be to focus on representing their published analyses in our article, rather than focusing on the primary sources that we personally consider particularly interesting. In my experience, this is something that newbie editors – especially ones that aren't familiar with the secondary literature – generally don't accept, and even many seasoned editors with many thousands of edits to their name often find hard to let go of. But the result is that WP becomes a vehicle for original research, and a secondary source; so I think a corresponding statement here would be useful. Jayen466 18:54, 5 January 2009 (UTC)

I'm not talking about "pet ideas" at all. If that's what you're intending to get at with this proposal, why not take it where it belongs: WP:WEIGHT. Anomie⚔ 00:57, 6 January 2009 (UTC)

This is not just about plot summaries. If you like, we can rephrase it more generally in terms of citing primary sources. So, are we in agreement that the following reflects the spirit of the project: If there is a substantial body of secondary literature available on a body of primary sources, then Wikipedians should not choose themselves what to cite from the body of primary sources, but should follow the selections found in the secondary literature?

Or do you think that Wikipedians are as entitled as published authors to identify statements of importance and interest to the Wikipedia reader in primary sources, and that it is okay, even desirable, if the focus and priorities of Wikipedia articles differ from, or are more detailed and comprehensive, than those found in the existing body of reliable sources discussing the relevant body of work? Jayen466 10:45, 5 January 2009 (UTC)

If you're summarizing the plot, you should cite whatever statements are necessary to summarize the plot; often, you won't need any direct quotes and you may not even need an explicit citation. If you're highlighting quotes for commentary or analysis, you need secondary sources anyway to support the commentary and analysis so the issue shouldn't even come up. And if you're highlighting quotes just to highlight quotes, see WP:TRIV. Anomie⚔ 17:29, 5 January 2009 (UTC)

It's a fallacy to assume that you need commentary or analysis to communicate a POV. For example, I could quote or descriptively summarise 10 passages from the Bible, without commentary, that would leave you with a definite impression as to what kind of document it is. Someone else might pick 10 quotes that leave a totally different impression. Likewise, I could give you 10 quotes suggesting that Shakespeare or Goethe, say, viewed women as inferior to men, or any other agenda you might care to mention. As I said above, c'est magnifique, but ... the point is, even though such primary-source quotes are verifiable, and purely descriptive, the selection itself from the overall reservoir of available statements is a kind of OR. Hope you see what I mean. Jayen466 18:28, 5 January 2009 (UTC)

The act of selecting sources for an encyclopedia article inherently involves judgment. If one is using a detailed secondary source for the plot, one typically does not include every quotation that it includes. As soon as you begin selecting material for an article from the available sources, a degree of subjectivity is involved. I think plot summaries are in fact exceptions to the role of preferring secondary sources--most secondary sources for the plots of television in particular are extremely inferior teasers, and the only justification for using them would be if the original work were not available. DGG (talk) 18:27, 5 January 2009 (UTC)

You could pick 10 reviews in secondary sources to write a completely biased article, too. Anomie⚔ 00:57, 6 January 2009 (UTC)

Yes, but there is a difference. Those ten secondary sources, all taking a particular POV, may still have a place in the article once it reaches FA status; all that's necessary is that all the other secondary sources that take different views need to be added as well, to balance the article. With primary-source quotes, however, these simply should not be in the article at all if (1) there is a substantial body of secondary sources and (2) none of these mention that primary-source passage suggesting that Goethe was gay, Avicenna was an alcoholic, or Mark Twain was a racist. Jayen466 01:26, 6 January 2009 (UTC)

• I've kicked off a discussion on the NPOV talk page, referencing this one. Jayen466 17:47, 6 January 2009 (UTC)

• Further to the above, I've posted a proposed rewrite of WP:Undue at NPOV talk. Would be interested in your feedback. Cheers, Jayen466 00:24, 9 January 2009 (UTC)

Burden of evidence?

Now that most of Phil Sandifer's issues with the "primary sources" section have been resolved, I want to split off and focus on his proposal that we add a statement essentially saying "Don't remove sourced material unless you actually think the source does not clearly support it." I think this really goes to the question of "Burden of Evidence.

I think most of us would agree that if an uncited statement seems to be OR, the burden of evidence rests with those who want to include the statement. They have to provide sources that back the statement, to demonstrate that the statement isn't OR. In this burden of evidence, we are in sync with WP:V.

Phil's proposal focusses on the next step... when a statement is cited, what do we do if someone thinks that statement is OR. I think Phil is essentially raising the question: where does the NOR burden of evidence lie if something is cited? Does the challenger have to demonstrate that it is OR? does the person who wants to include the statement have to demonstrate that it isn't OR? Or is the burden some combination of the two?

Of course this leads us to a more fundamental question: When it comes to NOR, is there a "Burden of evidence" (similar to what is in WP:V) and, if so, should we lay it out in the policy? Blueboar (talk) 18:10, 6 January 2009 (UTC)

I think the statement, "Don't remove sourced material unless you actually think the source does not clearly support it," was a quick toss off, so I don't know what purpose something like this (as it is worded now) would have. When will an editor say, "yeah, I think the source clearly supports the claim but I'm going to call it OR and remove it anyway?" If this is an argument to shift the burden, then no.......that takes us into negative proof. Say a claim is cited to Gibbon. Another editor says, "the claim's not clearly supported, I dispute it." The first editor can't say, "You have the burden of proof, so show me how the claim isn't supported." Professor marginalia (talk) 19:10, 6 January 2009 (UTC)

Actually, I think both editors have some degree of burden... When a citation has been given, the resolution of OR issues should always be a discussion ... the challenger needs to explain not only that he thinks the source does not support the statement, but also why he thinks this. On the other hand, the editor who wants to keep the statement should respond by explaining why he/she thinks the claim is supported. Blueboar (talk) 19:26, 6 January 2009 (UTC)

I think burden of proof might be the wrong phrase to describe this. It seems to me that what's at issue in these situations is not a dispute over the burden of proof, because we are fairly clear on that - we want to accurately describe sources. The threshold, then, is whether there is a consensus that a source is described accurately.

In the absence of a dispute, the assumption is that there is a consensus. The status quo is always assumed to have consensus.

The real question, then, is what sorts of arguments do we take seriously in trying to formulate a consensus when there is a dispute. Or, more broadly, once we have a dispute about what a source says, how do we set about hashing it out?

To which the shortest form of the answer (which I'm hoping will suffice, because I'm tired and don't want to write up the long form) is "by carefully reading the source in question, in as much detail as needed, and providing evidence for claims about what it says." But one thing - and I think it's an important thing - is that we should argue this based on what the source says. One important thing is that this is an argument that is in part about affirmative meanings - that is, we need to get away from a very bad habit that I've seen of arguing about what some reader might potentially see in the source. Readings are not potential. Arguments about what a source says are hashed out by closely reading the source and figuring out what it says. If people genuinely disagree such that there is no clear and consensus description of the source, well then, there's no clear adn consensus description of the source to use. But we need to get away from "that's not what it says" and "someone might disagree that's what it says" and towards "here is what it says, and here is how I know." Phil Sandifer (talk) 19:32, 6 January 2009 (UTC)

I think arguing "that is not what it says" is quite valid in an OR discussion ... otherwise what you say all makes sense. But, do we really need to say this in the policy? I don't think so. You ask, "what sourts of arguments do we take seriously in trying to fomulate a consensus when there is a dispute?" I don't think this is something we can or should outline in a policy. Because each OR debate is going to be a bit different, depending on the statement under dispute, the source being used to support it, and the editors involved in the dispute. "How do we set about hashing it out?" ... my answer to that is: We discuss the issue on the talk page, asking for third party opinions if needed. Blueboar (talk) 20:19, 6 January 2009 (UTC)

Well, my point is that "that's not what it says" should be, perhaps, replaced with "no, it doesn't say that, it says this." But I do think that the problems in question do come from this policy - with its focus on imaginary "reasonable, educated" people and a priori distinctions between interpretation and description instead of on the mechanics of practical consensus. Which is, essentially, what I would like to specify - that this policy is not about imagined and hypothetical readings of sources. That if you think a reading of a source is wrong, be prepared to argue what you think it does say, not what some imagined person might think it could say. Phil Sandifer (talk) 20:34, 6 January 2009 (UTC)

OK, I can agree that saying "no, it doesn't say that, it says this" is a better way to approach the issue... but again, I don't think this is something we can or should deliniate in a policy. Some challenges are going to be poorly worded, some are even going to be poorly thought out. That isn't something we can legislate. The only way to deal with challenges is through discussion and consensus. Blueboar (talk) 21:00, 6 January 2009 (UTC)

Under normal circumstances I would agree with you, but since so much of this policy has what seems to me very poor wording that encourages a focus on theoretical readings instead of actual ones, I think it is, in this case, more necessary than it might otherwise be. I suppose I don't see why you think a note to the effect of "Unlike unsourced information, which can be removed on sight, disputed sourced information should be discussed with an eye towards trying to come to a consensus about what the source says" is problematic. Phil Sandifer (talk) 23:29, 6 January 2009 (UTC)

It isn't that I think it problematic, its that I think it is completely unnecessary. Blueboar (talk) 03:41, 7 January 2009 (UTC)

Blueboar, I oppose adding a statement like "Don't delete stuff that is actually supported by the source" -- especially in the absence of a real-world/can't be solved any other way problem that requires it. This creates direct conflict with WP:DUE, which is going to be a nightmare with wikilawyers. Furthermore, I don't really know how we can say "if you know the source supports the material, then don't delete and tell lies in your edit summary about how it's not supported by the source" without assuming bad faith. WhatamIdoing (talk) 00:51, 7 January 2009 (UTC)

I think it's less about assuming bad faith than about assuming brain-searing stupidity. Which is, sadly, generally a good assumption. Phil Sandifer (talk) 00:59, 7 January 2009 (UTC)

Last I checked Wikipedia does not have a WP:No brain-searing stupidity policy. Occasionally other editors are going to do or say things that you or I might think are stupid. We can not write policy to try to prevent that. Blueboar (talk) 03:45, 7 January 2009 (UTC)

I'll start by saying that we should restrict this discussion to removal of sourced material because it is original research. Sourced material may be removed for many reasons, up to and including mere editorial discretion ("too detailed for an encyclopedia article", "not sufficiently relevant", etc.) That said, no matter why someone removes sourced material, they ought to justify and support their decision; if they believe the source does not support the material, they should either verify that the source does not support the material, provide evidence that the source is not reliable, or at least provide reasonable justification to support the claim that the source does not support the material (e.g., "this source was published in 1934 before the transistor was even invented"). The person making the claim would then be able to retort, and stronger justifications would be more difficult to argue against. There is no one-size-fits-all rule here. Dcoetzee 03:48, 7 January 2009 (UTC)

RE "There is no one-size-fits-all rule here" : Agreed. ... Kenosis (talk) 04:21, 7 January 2009 (UTC)

I think both Phil and I agree with what you say... the issue is that Phil thinks we need to include something saying all that in the policy, while I think it is essentially unnecessary. Blueboar (talk) 04:18, 7 January 2009 (UTC)

I agree. ... Kenosis (talk) 05:50, 7 January 2009 (UTC)

I also think Phils' proposal is unnecessary, and I'm amused at Crotalus Horridus' redirect on Blueboar's used-to-be-a-redlink. WhatamIdoing (talk) 21:37, 7 January 2009 (UTC)

Me too :>) Blueboar (talk) 05:21, 8 January 2009 (UTC)

Here's my big concern, and why I'm unpersuaded by the unnecessary argument.

1. People interpret policy with a harsh literalism, and an explicit reminder that a sourcing dispute needs to be settled via careful discussion of the source, not policy wielding, is an important hedge against that.
2. The policy as it stands has some really problematic wording that comes across as making absolute claims and rules about the nature of a reading of the source. Stuff like "interpretive" and "descriptive," or the "a reasonable, uneducated person" test give the impression that the way to judge if something is OR is through a series of thought experiments. I know that moving NOR away from that (which would be my first choice) is going to be a tortuous, difficult shift. In my ideal world, we'd remove all of the language about "descriptive" claims and the like, replacing it with "All cited claims should be clearly backed up by the source cited. Claims that are not clearly backed up by the source are original research. Disputes over what sources say should be dealt with by close, careful discussion of the source. In the event that the source leads multiple editors to multiple readings of it on a given point, the source is insufficiently clear to be used for any definitive statements on that point."

That could, in fact, replace a large swath of this policy. And I'd like to see it done. But I get the feeling that pushing on that front would be met with extreme hostility.

I mean, thoughts? I want to make this discussion move smoothly and sensibly. But the fact of the matter is, I think this policy has some serious failures to describe a usable or accepted practice, that it has fallen very, very far from its original intent and consensus, and that it needs serious reform. I'm willing to leave it at enshrining some language that moderates against the policy's worst instincts, or I'm willing to spend a long time on this page trying to fix it. Since you folks seem to be the people I'd be working with, what would you prefer? Phil Sandifer (talk) 15:42, 8 January 2009 (UTC)

What would I prefer? I would prefer to hive off the PSTS section entirely, and put it in its own guideline or policy... as I think it is a distraction from the main point of the policy (which is that regardless of the type of source, we should not include our own original thinking about either the source or the topic in our articles.) However, I am well aware that in the minority on this point... I accept that the consensus is to continue to have the PSTS section in this policy. Given that acceptance, I would prefer to leave the policy as it is, since no one (except Phil) seems to think there is a any sort of huge problem here. Blueboar (talk) 16:00, 8 January 2009 (UTC)

It seems more accurate to say that the preponderance of rules lawyers and pedantry have driven large swaths of editors away from policy formation. Try having a discussion like this on the mailing list (a good place to find plenty of long-time Wikipedia users who have migrated away from policy discussions) and you'll get a very different perspective. But I'm not even going to try fixing the problem of why experienced Wikipedians have given up in disgust on policy formation.

In any case. I'm committed to fixing this problem. How would you like to go about it? Phil Sandifer (talk) 16:51, 8 January 2009 (UTC)

I agree with Blueboar. There is no problem to fix. Leave the policy as it is. The focus of your objection to this section of the policy has shifted 3 or 4 times now. And now you object because it encourages "policy wielding" rather than "discussion"? Why wouldn't this be a conveniently general rationalization for weakening or eliminating any and every policy in wikipedia? We've discussed this at length, and I am not persuaded at all there is support for changing it, or that there is a need to do so. The examples you've offered aren't doing it for me. I don't think they're demonstrating any need for policy change. For example, you characterized the policy as "very problematic" for The Yellow Wallpaper, but the article indicates at least a dozen secondary sources which do resolve the problem. Jacques Derrida was another one example--secondary sources, one, but two, he is notoriously difficult to follow, which even fellow philosophers concede to. Any loosening of the restrictions as far as Derrida is concerned would invite far less credible edits and far more contentious dispute on the talk page-certainly neither being a net plus to the encyclopedia. Examples given from science and Critical Inquiry have been offered as hypothetically problematic in terms of the policy, but they're not convincing to me. And now you're claiming, without examples, that the policy serves to discourage discussion? I've dealt with original research disputes many times. Most of these disputed edits--I will say all of those added by editors who are willing to assemble so much as a loosely credible case for keeping the edits--end up on the talk page and are discussed, at length most of the time. There is no problem except when there aren't sources to be had about a subject which say what the editor wants to say about the subject. And that is a real problem, not a hypothetical. I repeatedly see editors stray into original research with primary sources by taking examples from the source to illustrate or bolster an argument or opinion which is not made in the source. Professor marginalia (talk) 18:56, 8 January 2009 (UTC)

My objection hasn't shifted a jot, though perhaps your understanding of it has been. Phil Sandifer (talk) 19:06, 8 January 2009 (UTC)

"I repeatedly see editors stray into original research with primary sources by taking examples from the source to illustrate or bolster an argument or opinion which is not made in the source".: Well put. If the policy has a shortcoming, it's that it does too little to explain to editors why exactly this sort of thing is a credibility drain for a project aiming to be a serious encyclopedia, one that reflects the most reliable sources out there. Jayen466 19:35, 8 January 2009 (UTC)

To PS: "The big problem is a false opposition between 'interpretation' and 'description'"[2] ; the "specialist knowledge" clause was made without discussion or consensus [3] ; the "specialist knowledge" clause violates NPOV [4]; policy is unclear that "descriptions of sources that nobody disagrees with are OK" [5]; and policy sounds like "absolute claims and rules" about how to use a source, leading to "policy wielding" where "careful discussion of the source" is preferable [6]. It's clear that section of the policy doesn't sit right with you-but even after all this discussion, it's not clear to me that any of these are well substantiated concerns that require any change to it. Professor marginalia (talk) 20:05, 8 January 2009 (UTC)

Lets get down to brass tacks here... does anyone other than Phil see a need to address any of this in the policy? Blueboar (talk) 20:35, 8 January 2009 (UTC)

Yes, because a poll of the four people who have been participating in this discussion is *clearly* useful in some fashion. Phil Sandifer (talk) 21:04, 8 January 2009 (UTC)

Sorry, perhaps I was not clear... Anyone OTHER than Phil? Blueboar (talk) 21:11, 8 January 2009 (UTC)

Perhaps I was equally unclear - I don't think that, in a discussion consisting of four people, "you're the only one who thinks that" is really a very effective put-down. Now if you want to get other people involved substantively in the discussion - that is, engaging with points, responding to arguments, etc, fine. But as long as it's you, marginalia, and cameos from Kenosis, no, you do not have the weight of consensus against my arguments, and trying to shut them down that way is, frankly, bullshit. Phil Sandifer (talk) 23:25, 8 January 2009 (UTC)

That's a fairly infuriating comment. We've made every attempt to discuss it you. And I put together an RFC to bring in others, posted notices everywhere, including to past and current contributors' talk pages[7], seeking input for a concern of yours with this clause, not mine or blueboar's. Did it stimulate any agreement with you? The only other explanation given here for a lack of participation in the discussion, besides perhaps you're alone in seeing the problems you have with it, was the WP:TLDR complaint made against you, twice. Professor marginalia (talk) 00:16, 9 January 2009 (UTC)

And, if I may point out, that worked - the specific issue that I was focusing on, and that the RFC focused on, was the specialist knowledge clause. Which is now no worse than the rest of the policy. And now I move on to other issues I have with the policy. As for your TLDR suggestion, frankly, if anyone sincerely thinks that I've been unnecessarily verbose, honestly, fuck them. I've been carefully laying out as clear discussions of complex issues as I can, taking a lot of time to do it. But you know what? It's a complex issue. The line between summary and original research is complex. And people who want to reduce it to one line bromides and declare the moment that it gets at all complicated that I'm a POV pusher who wants to insert academic OR into articles can quite honestly fuck themselves.

If any of you spent half the effort you've spent trying to shut me up actually reading my arguments and discussing them instead of going "lalala consensus so we don't have to discuss go away," we'd get somewhere. But, frankly, you - and here I'll single you out - have spent the whole of your effort trying to shut me up. And then you have the gall to call my posts infuriating? Hilarious. Beyond comedy. Phil Sandifer (talk) 00:25, 9 January 2009 (UTC)

Single me out for trying to "shut you up"? What? I demand you show me where I've done anything of the kind. This is unfair in the extreme. Earlier today in this thread you protested that I misunderstood you to say that your complaint with the clause "had shifted."[8] Now you argue back at me that you've now "moved on to other issues"? The TDLR wasn't my suggestion, it was the objection given from other editors to say why they weren't responding to your comment. You've got all the bases covered. Those do drop in to respond are dismissed as "cameos", those that stay and respond to your followups, as I have, are brushed off as "trying to shut you up" and those that pointedly tell you why they won't respond, you say "fuck em". And those that aren't here at all--they're the editors who likely agree with you. Clever. Professor marginalia (talk) 00:47, 9 January 2009 (UTC)

Phil, I did not intend my initial comment to single you out (looking at my repetition, I can see how you might get that impression. it was meant as humor and I appologize if that humor did not come across well). Nor do I intend to "shut you up" or limit anything to just the four people who are actively engaged in the conversation. It was meant as an invitation for others to join the converstation. We know how you feel about this, and we know how I, Professor marginalia, Jayen, Kenosis, WhatamIdoing and the few others who have commented feel. I want to hear from someone who hasn't said anything yet. Blueboar (talk) 01:09, 9 January 2009 (UTC)

My apologies for misunderstanding. Unfortunately, I fear this thread (largely through my doing) has become a bit toxic. I've started one below to see if it's possible to generate a clear sense of my issues here - since I don't get the sense that I've been well understood. Perhaps I'm wrong and I really am off in the wilderness on this one, but I think there is some real tightening that we could do that would make this policy both less unclear (as I think it's poorly used somewhat often) and also more potent. But let's see if we can at least get on the same page about what's at issue - which I don't think we are. Whoever shows up. :) Phil Sandifer (talk) 01:20, 9 January 2009 (UTC)

Mathematical proofs

Would this policy permit mathematical proofs, as long as the starting statements are sourced and each step taken thereafter uses nothing but simple mathematical axioms (distributive law, commutative law, etc.)? --Oboeboy (talk) 16:50, 8 January 2009 (UTC)

Original mathematical proofs are a form of original research. At present, the policy states: "The "No original research" rule does not forbid routine calculations (e.g. adding or subtracting numbers, rounding them, calculating percentages, converting them into similar units, putting them on a graph, or calculating a person's age) that add no new information to what is already present in the cited sources." How far this gets stretched will perhaps obviously be subject to discussion and debate. More than a few articles requiring math calculations (as differentiated from proofs) are arguably somewhat out of control. Someone brought up Loop quantum gravity recently, which quite arguably might be an example of one that's gotten a bit out of control. Similarly, articles like Size of the universe have some highly debatable calculations, although those calculations were largely sourced (which is also an issue of whether the sources are WP:V#Reliable_sources, discussed at greater length in the guideline page WP:RS).
...... Original proofs, however, even when using accepted rules of proof, are an entirely different matter because they're not routine and they do add very significant additional information not present in the source(s). Speaking as just one WP user familiar with the core content policies, I would advocate that any mathematical proof should be cited to a reliable source that presents the same proof, rather than trying to provide a customized proof formulated by a WP editor. ... Kenosis (talk) 17:53, 8 January 2009 (UTC)

NB: With respect to the situation at Loop quantum gravity, one would now need to look through the history to find the issue I referred to above. I've removed the block of original research posted in the section previously titled "Diffeomorphism invariance and background independence", and replaced it with what I suppose is an equivalently imperfect summary-style section, presently at Loop quantum gravity#General_covariance_and_background_independence. ... Kenosis (talk) 23:00, 8 January 2009 (UTC)

No. Jayen466 19:39, 8 January 2009 (UTC)

It comes down to this: Mathematical proofs must come from a reliable published source, like any other type of information in Wikipedia. Blueboar (talk) 20:40, 8 January 2009 (UTC) ...

Oboeboy is posting as a reaction to Talk:Goldbach's conjecture#Polignac's Conjecture. He has been edit warring to add his own unsourced alleged proof. 5 different editors have reverted, not only because it's original research but also because it's trivially false. An important reason for the policy against original research is to avoid exactly what you are trying to do. Please stop trying to come up with excuses to add this alleged proof again. You have already tried to invoke WP:IAR, claimed that removing the proof is vandalism so WP:3RR doesn't apply when you readd it, and now a vague post here without revealing that many editors agree your simple "proof" is simply false. The article is being watched and your "proof" will not be allowed so you are wasting your time. PrimeHunter (talk) 00:38, 9 January 2009 (UTC)

Failing the "directly related" test - Examples please

The policy states:

Citing sources and avoiding original research are inextricably linked: to demonstrate that you are not presenting original research, you must cite reliable sources that are directly related to the topic of the article, and that directly support the information as it is presented.

I find "directly related" to be quite vague here and it would be nice to have at least one example in the policy of a case that fails the "directly related" test. Can someone give a clear example where an editor violates the OR policy by citing sources that are NOT directly related to the topic of the article? Thanks!

--Phenylalanine (talk) 14:55, 3 January 2009 (UTC)

It will have to be out of the history of an article, because I hope that any such text based on such a citation is deleted from the article :-) Often it has to do with WP:SYN and such examples occur frequently in Genocides in history. An editor assumes that because tens of thousands have died the event must have been a genocide. But it is quite possible for tens of thousands to be killed and it not to be considered a genocide in reliable sources. A good example is this is Genocides in history: Congo where it is agreed millions died and there are sources cited that said so, but no reliable sources claim it was a genocide, see Talk:Genocides in history/Archive 6#Belgium and the Congo --PBS (talk) 15:50, 3 January 2009 (UTC)

Thanks Philip. That is a good example of failing the standard "directly support" test. When applied to the logical flow of the text, failing the standard "directly support" test amounts to "original synthesis". What I'm looking for is a specific case that fails the "directly related" test, but not necessarily the standard "directly support" test. --Phenylalanine (talk) 16:28, 3 January 2009 (UTC)

I looked over your example again and I agree that it exemplifies a case that fails the "directly related" test. --Phenylalanine (talk) 02:24, 5 January 2009 (UTC)

I think they are opposite sides of the same coin. Blueboar (talk) 18:32, 3 January 2009 (UTC)

Can you provide an example of a specific case that fails the "directly related" test, but not the "directly support" test? Cheers, Phenylalanine (talk) 18:39, 3 January 2009 (UTC)

Phen, why don't you explain your actual dispute (either here or at NORN) instead of talking about hypotheticals? WhatamIdoing (talk) 02:53, 4 January 2009 (UTC)

I don't have a specific dispute in mind. There have been efforts on this page to clarify the meaning of "directly related", but all such attempts have failed so far. Based on previous discussions, my understanding of "directly related" is that, irrespective of the sources cited, if content appears to be relevant to an article (and the sources directly support the content as presented), the sources automatically pass the "directly related" test. Am I not correct? --Phenylalanine (talk) 04:40, 4 January 2009 (UTC)

The most recent discussion on this is in Archive 38... (here) This may inform you as to the intent of this passage. Blueboar (talk) 16:32, 4 January 2009 (UTC)

The example in WP:SYN, though obviously made-up, is a pretty good example of failing the directly related test while passing the directly support test. The source given does actually directly support the claim that using a source indirectly is not plagiarism; but as the source is not directly related to the case in question, it is still original research. TSP (talk) 16:42, 4 January 2009 (UTC)

Let's take a closer look at the example on the policy page. Suppose we remove the editors opinion: "If Jones's claim that he consulted the original sources is false, this would be contrary to the practice recommended in the..." (which does not pass the directly support test), and suppose each of the sentences individually pass the directly support test (the source cited directly supports the information in the sentence as it is presented). Assuming that the first paragraph passes the directly related test (the sources are directly related to the topic of the article), do the other relevant sentences also pass the test, assuming that the sources do not specifically comment on the dispute? (The article is about "Jones".)

Smith says that Jones committed plagiarism by copying references from another book. Jones denies this, and says it's acceptable scholarly practice to use other people's books to find new references.

In this regard, the Harvard Writing with Sources manual requires citation of the source actually consulted. The Harvard manual does not call violating this rule "plagiarism". Instead, plagiarism is defined as using a source's information, ideas, words, or structure without citing them.

Let's change the example a bit without modifying the assumptions made above. Let's suppose that Jones, for whatever reason, has not yet denied Smith's claim that he has committed plagiarism and, so, we cannot cannot report this. Assuming that Smith's claims are noteworthy because they have received broad media coverage, isn't the second paragraph necessary, at least in part, to uphold the neutral point of view?

Smith says that Jones committed plagiarism by copying references from another book.

In this regard, the Harvard Writing with Sources manual requires citation of the source actually consulted. The Harvard manual does not call violating this rule "plagiarism". Instead, plagiarism is defined as using a source's information, ideas, words, or structure without citing them.

(1) If you believe that the second paragraph fails the directly related test (the sources are not directly related to the topic of the article) in both examples (since the sources do not specifically comment on Smith's claims), but is required per WP:NPOV in the second example, then you must believe that Wikipedia policies may from time to time conflict. This position is barely tenable.

(2) If, on the other hand, you believe that the second paragraph passes the directly related test (the sources are directly related to the topic of the article) in both examples, despite the fact that the sources do not specifically comment on Smith's claims, and you believe that the second paragraph is required per WP:NPOV in the second example, then you are forced to concede that "original synthesis" comes down to failing the "directly supported" test, i.e. when applied to the logical flow of the text, failing the standard "directly support" test is a sufficient and necessary condition for "original synthesis". The problem with approach is that it opens a huge loophole: irrespective of the sources cited, if content is relevant to an article (and the sources directly support the content as presented), the sources automatically pass the "directly related" test. This obviously opens the door to original research: information that sheds new light on an article topic by making connections, not mentioned in the literature on the topic, that might not be immediately obvious to researchers working on that topic, such that these connections might suggest new research conclusions or approaches.

(3) Another way to look at the issue is that the second paragraph fails the directly related test and, consequently, because WP:NPOV is based on WP:NOR, the second paragraph is not required by the WP:NPOV policy in the second example. This my preferred approach and it is based on a logical extension of the WP:SYN rule to implied or suggested conclusions:

Statements verified by sources which do not refer to the subject-matter of the article are deemed to constitute original synthesis when such statements clearly imply and logically entail non sourced, noteworthy conclusions about a "topic" [in this case, Smith's claims] related to the article subject, or when such statements evidently suggest, but do not logically entail, non sourced favorable or unfavorable assessments about that that particular related "topic".

Here, the term "subject-matter" is broad enough to include relevant information from sources that do not necessarily mention the article name itself or synonyms thereof, but avoids the loophole in the second approach. This approach also allows supplementary information from sources that do not refer to the subject-matter of the article, provided that the information is deemed to add value to an article, in order to clarify places, people or things mentioned in the article, without advancing a position or point of view with respect to any aspect of the article topic (Examples of this sort of information: [9] [10][11]).

--Phenylalanine (talk) 21:12, 4 January 2009 (UTC)

The Smith and Jones example is also questionable. It too seems to have been added by SlimVirgin in reference to one specific argument (added argument].) People have been puzzled by this before, more than once.

I find the example nonsensical. Applying a factual definition isn't original research; the problem is that we have no reason to believe that a factual definition from the Chicago Manual of Style is relevant in the first place. This means that the problem is correctly described as unreliable sources--not original research. Ken Arromdee (talk) 22:48, 4 January 2009 (UTC)

A and B, therefore C.

• A = Smith says that Jones committed plagiarism by copying references from another book.
• B = The Harvard manual does not call violating this rule "plagiarism".[source does not comment on dispute]
• C = Therefore, if Jones copied references from another book, he did not commit plagiarism according to the Harvard manual.[source does not comment on dispute]

In my opinion, this is original synthesis, since we are putting A and B together to arrive at a new position (C). --Phenylalanine (talk) 00:18, 5 January 2009 (UTC)

Indeed, an editor's interpreting the Chicago Manual of Style's guidelines, and using them to decide whether or not an author plagiarized, in order to bolster a debate about exactly that issue, is about as clear an example of WP:SYNTH as one could imagine. Interpreting style guidelines and applying them to real-life issues in order to support one side of an argument is not "applying a factual definition". Jayjg ^(talk) 01:33, 5 January 2009 (UTC)

But in this particular case, the Chicago Manual of Style's guidelines are unambiguous enough that they can be applied without any nontrivial interpretation. Interpreting style guidelines is OR, but applying style guidelines without interpretation isn't OR. The only question is whether you have a reason to apply them--there's no question what result you get if you apply them. In other words, the problem is unreliable source rather than original research. 208.89.102.102 (talk) 07:03, 5 January 2009 (UTC)

Ken, applying a non-obvious definition is original synthesis if the source does not apply the definition to the specific real-world case. In other words, if you feel the need to reference a dictionary to apply a definition, your doing original research. --Phenylalanine (talk) 01:45, 7 January 2009 (UTC)

Here's an example happening right now. In the article Anti-Jewish violence in Poland, 1944–1946 an editor keeps trying to insert material about Poles who hid Jews during the Holocaust.[12] The editor is clearly trying to make an argument of sorts - "even if some Poles did attack Jews, others hid them." The problem is, is that material "directly related" to the topic of the article? Those opposing the insertion of the information insist that it is not; as evidence, they point out that no reliable sources have linked the two subjects. Clearly the sources being used do not. Jayjg ^(talk) 01:33, 5 January 2009 (UTC)

Excellent example.--Phenylalanine (talk) 02:16, 5 January 2009 (UTC)

That's an example of someone trying to include information that isn't related to the topic of the article, not this obscure issue about the source being "directly related". Anomie⚔ 03:55, 5 January 2009 (UTC)

I don't understand your comment. The editor believes the the information is indeed related to the topic of the article. The problem is that the sources aren't "directly related", which is what helps us then say with authority that the information is not related either. Jayjg ^(talk) 04:38, 5 January 2009 (UTC)

If that's the meaning of this "directly related to the topic" statement, I'd have to say it's rather widely ignored and IMO not something that should even be included here (WP:NPOV would be a better place, and the essay WP:COATRACK talks about that sort of thing). Articles often contain background statements that need to be sourced. For example, an article on a historic event or scientific discovery may need to provide a brief historical background, and the statements in that background may need citation to sources that have nothing to do with the topic (except that the topic occurred against that historical background). Or an article on a fringe theory might contrast it with the mainstream threory, but the mainstream theory sources may completely ignore the fringe theory and thus not be directly related to the topic (only to the larger issue that both theories theorize about). And how many sources in various biographies relate only to the subject's works rather than directly to the subject himself? The Polish apologist described above is pushing a POV by adding irrelevant text to an article, but I'd hardly call it "original research". Anomie⚔ 17:11, 5 January 2009 (UTC)

We're going to have settle this dispute once and for all. I think the debate can be summed up as follows:

Editor A: WP:NOR should be interpreted less stringently to accommodate the WP:NPOV and WP:PCR rules.

Editor B: No, WP:NPOV and WP:PCR should be interpreted less stringently to accommodate the WP:NOR rule.

I think both sides have their pro and cons. Anomie describes the pros of editor A's approach and the pitfalls of editor B's approach. However, Editor A's interpretation is a slippery slope and it opens a huge loophole: irrespective of the sources cited, if content is relevant to an article (and the sources directly support the content as presented), the sources automatically pass the "directly related" test. This clearly leads us to original research: information that sheds new light on an article topic by making connections, not mentioned in the literature on the topic, that might not be immediately obvious to researchers working on that topic, such that these connections might suggest new research conclusions or approaches. On the other hand, Editor B's approach avoids this loophole, but perhaps fails by not giving enough force to the WP:NPOV and WP:PCR rules. Is there an option C that avoids the pitfalls in both approaches? --Phenylalanine (talk) 01:49, 6 January 2009 (UTC)

I would also like to see some examples where information is relevant to the topic of an article and the sources directly support the information as presented, but where these sources are not directly related to the topic of the article. --Phenylalanine (talk) 02:24, 5 January 2009 (UTC)

How about this one. There was an addition made to the article on the Butt Report which changed a sentence from "In response to the concerns raised by the Butt report, Cherwell produced his dehousing paper..."(Longmate The Bombers: The RAF offensive against Germany 1939-1945 p. 126), to "After the Butt report and Area bombing directive had been issued, Cherwell ...".[13] As it happens no source was given for the date of the "area bombing directive", but it is trivial to find one which the editor could have included. However while it is true that the area bombing directive was in part worded in response to the Butt Report, there is no evidence presented that Cherwell produced his dehousing paper as a partial result of the area bombing directive. Although it is true that the dehousing paper was produced after the directive, it is also just as true that it was produced after Portsmouth F. C. won the FA Cup in 1939. So the area bombing directive is directly related to the topic of the article (it was one of the outcomes of the Butt report) but it (a date citation for the directive) does not 'directly support the information as it is presented as an addition to that sentence. --PBS (talk) 17:06, 8 January 2009 (UTC)

Thanks for the example Philip, but this is not the kind of case I was referring to. As you indicate, this is an example where (1) the information is effectively relevant to the topic of the article but where (2) the source cited does not directly support the information as it is presented. What I am specifically looking for is an example where (1) the information is effectively relevant to the topic of an article and (2) the source directly supports the information as it is presented, but where (3) the source is not directly related to the topic of the article. Cheers, Phenylalanine (talk) 01:13, 9 January 2009 (UTC)

But the phrase are "you must cite reliable sources that are directly related to the topic of the article, and that directly support the information as it is presented." In the above, the "Area bombing directive" would be directly related, but it fails to support the information as presented (in that sentence). As there are two tests there are four possible outcomes:

Directly related source	supports the information as presented
yes	yes
no	yes
yes	no
no	no

Perhaps it needs changing to "you must cite reliable sources that directly support the information presented in the article, and that information must be pertinent to the topic of the article." but I think such a change needs much more discussion. --PBS (talk) 14:08, 10 January 2009 (UTC)

I think such a change would be problematic for two reasons:

• (1) It opens the door to genuine original research: information that sheds new light on an article topic by making connections, not mentioned in the literature on the topic, that might not be immediately obvious to researchers working on that topic, such that these connections might suggest new research conclusions or approaches.
• (2) It introduces an untenable double standard, i.e. if you wish to make a point about a topic without a source making that specific point, you may do so, just so long as you do not state that point explicitly (WP:SYN).

--Phenylalanine (talk) 15:22, 10 January 2009 (UTC)

And again

/sigh

I apologize for my frustration here and above. I've found this, without question, the absolute most frustrating policy discussion I've engaged in here. I'm not sure why - perhaps I'm failing to be clear. Perhaps the discussion is too broad and derails easily. Perhaps there just aren't enough viewpoints coming in.

In any case. I will try again. I will attempt to be as succinct as possible. In return, if you will, let us treat this as a new discussion. A new leaf turned over. Yes?

The policy says:

Information in an article must be verifiable in the references cited. Article statements generally should not rely on unclear or inconsistent passages nor on passing comments. Passages open to multiple interpretations should be precisely cited or avoided. A summary of extensive discussion should reflect the conclusions of the source's author(s). Drawing conclusions not evident in the reference is original research regardless of the type of source. It is important that references be cited in context and on topic.

Let me ask first: What does that paragraph allow that is forbidden by this paragraph?

Without a secondary source, a primary source may be used only to make descriptive claims, the accuracy of which is verifiable by a reasonable, educated person without specialist knowledge. For example, an article about a novel may cite passages from the novel to describe the plot, but any interpretation of those passages needs a secondary source.

That is, what is newly forbidden by the primary sources section that was not already forbidden for all sources? If we can have some consensus on that question, it will be easier for me to specifically identify the problem at hand, and whether it applies to primary sources or to all sources. Phil Sandifer (talk) 01:11, 9 January 2009 (UTC)

I don't think they say much of anything different in intent... the second paragraph (from PSTS) is a reinforcement of what was stated in the first paragraph. I don't think one allows something that the other forbids. I think the difference is that the two paragraphs approaches the same issues from different directions... The first from the direction of the statement being made, the second from the direction of the type of source being used. Blueboar (talk) 01:36, 9 January 2009 (UTC)

Is it fair to say, then, that the same restrictions exist for primary and secondary sources in terms of description vs interpretation, specialist knowledge, etc? Phil Sandifer (talk) 01:51, 9 January 2009 (UTC)

No... there are some restrictions that apply to primary sources that do not apply to secondary. I happen to think they should have the same restrictions (with the restrictions currently applying to primary sources also being applied to secondary), but at the moment they do not. Blueboar (talk) 02:07, 9 January 2009 (UTC)

No, it's not fair to say. The sentence that says "Without a secondary source, a primary source may be used only to make descriptive claims, the accuracy of which is verifiable by a reasonable, educated person without specialist knowledge " dovetails very effectively with WP:FRINGE and WP:Notability. It also buffers WP:V#Reliable_sources by insisting on the extra limiting factors in the use of primary sources. These extra limitations help prevent the misuse of primary sources. ... Kenosis (talk) 02:30, 9 January 2009 (UTC)

OK - then how can secondary sources be used? What is an example of a claim that can be made of a secondary source but not of a primary source? (Or, perhaps better yet, given that sources are primary for some purposes and secondary for others, what is a claim that can be made when it is secondary but not when it is primary?) Phil Sandifer (talk) 02:40, 9 January 2009 (UTC)

As it happens, I just ran into an example today, here. While there were several grounds for removal of the OR, WP:PSTS was one of them. Here's the diff, with secondary sources replacing the primary-source research. Possibly Professor Marginalia, who AFAIK is somewhat more active than I am in the WP "NOR patrol", has handy access to a few more examples. ... Kenosis (talk) 02:59, 9 January 2009 (UTC)

Phil, do you mean an example of a claim that can be made using a secondary source, but not a primary one? We get examples in animal rights articles a fair bit, where someone reads that Jesus, or someone from the same period, loved animals, and therefore adds that he was an animal rights supporter. That's a clear example of something that needs a secondary source, because good secondary sources are assumed to have informed themselves to a degree about the relevant literature, which means they can provide an educated overview that reliance on a primary source won't give you. SlimVirgin ^{talk|contribs} 03:26, 9 January 2009 (UTC)

Actually, what I'm more interested in here is an example of a claim that could be made about a given source if the source is being used as a secondary source, but could not be made if it is being used as a primary source. Phil Sandifer (talk) 03:32, 9 January 2009 (UTC)

I just gave an example. The very first sentence, whether in the form given here, or in the form I replaced it with here, would not be able to stand by citing to the cited primary sources or to any primary sources, at least not without being verifiable by a secondary source. In this case, there are a significant number of secondary sources available in support. If there were not such secondary sources available, then according to WP:PSTS the statement in the first sentence cannot be made. Rather, assuming the primary source is sufficiently noteworthy to discuss in the article, we can offer only a description of what the primary source says, in a form that can be checked by any "reasonable, educated person without specialist knowledge". An example of such a statement would be: "Source A defines 'diffeomorphism invariance' as follows: [fill in a quote or something that doesn't need a cosmologist to double check whether the statement reflects what's in the source]". Etc. ... Kenosis (talk) 04:45, 9 January 2009 (UTC)

An interjection here... Phil has used the phrase "claims that can be made of a primary/secondary source"... this is important. He is talking about a specific situation, and I would suggest a clearer wording would be "claims about the primary/secondary source itself".

The policy, on the other hand, takes a wider focus... it primarily talks about claims/statements that are cited to a primary/secondary source but are about something else. I am not sure if this distinction will impact our discussions... but it is worth noting. Blueboar (talk) 03:31, 9 January 2009 (UTC)

Phil, I don't know what you mean. SlimVirgin ^{talk|contribs} 03:46, 9 January 2009 (UTC)

Well, whether a source is a primary or secondary source depends on the context it's being used for - a book can be a secondary source for its topic and a primary source for itself. So the question is, what sorts of claims would be allowable when a book is being used as a secondary source that would not be allowable in a primary source. Phil Sandifer (talk) 04:01, 9 January 2009 (UTC)

Thinking about Phil's question a bit more... I think he is asking whether there is any difference in determining OR, or differnece in the NOR restrictions between when the subject of the article or section is a secondary source or when the subject of the article or section is a primary source. I would say not, there is no difference. This is because, Effectively, the "secondary" source becomes the primary source for that article or section.

To give an example... In an article on the 1911 Encyclopedia Britanica (which in other situations is actually an outdated tertiary source), the 1911 EB becomes the primary source for that article. Thus you should not cite the 1911 EB for analysis or interpretation of the 1911 EB or for conclusions about the 1911 EB (unless the analysis, interpretation or conclusion is specifically stated in the 1911 EB). To include such statements (assuming they are not specifically stated in the source) you need to refer to and cite a secondary source. Blueboar (talk) 03:57, 9 January 2009 (UTC)

Yes. This is a good example. So the question becomes, how do we distinguish between what is and is not allowed in those circumstances? Phil Sandifer (talk) 04:01, 9 January 2009 (UTC)

With good editorial judgment, an understanding of the basic principles, and if necessary, the friendly folks at WP:NORN. This is not nearly as hard as you're making it out to be. WhatamIdoing (talk) 05:34, 9 January 2009 (UTC)

In theory, you're right. In practice, many of our volunteer editors lack that understanding, and if they see something in the 1911 EB that strikes them as funny, not politically correct from today's vantage point, etc., they would happily write, "In its 1911 edition, the EB said that homosexuality was a mental illness caused by eating too much cabbage" or whatever, even if no published secondary source discussing the history of the EB had ever commented on that. And if you wanted to take that out, you'd have an armada of editors against you who find that quote just spiffingly interesting and cute. They'd tell you, "But it's verifiable and purely descriptive, what's your problem??" and suspect you of working for britannica.com. And if you went to NPOV/N or some such place, chances are a good half of the editors would echo, "But it's verifiable and purely descriptive, what's the problem? The 1911 EB did write that!" We should try to make policy clearer and explain to people what the problem is, and why. According to our remit, our analyses and examples should reflect the analyses and examples in the best sources out there. They should reflect not what is discussed and endlessly regurgitated on non-RS Internet sites and forums, but should reflect the writings of the most highly regarded scholars and experts on our topic. A subject matter expert reading our article should come away with the impression that the authors of the article know and consulted the relevant literature. If we don't do that, and instead rely on original research or a kind of "investigative journalism" to add "interesting facts" not discussed in the secondary literature, then we are not an encyclopedia, but something else. At present, we do not even tell editors that they should make an effort to familiarise themselves with the academic literature on the subjects they are contributing to. Something to that effect was in the policy once, briefly, and was speedily taken out again with the reasoning that it was "asking too much of editors" and "everybody should be able to contribute". Jayen466 10:35, 9 January 2009 (UTC)

Your example is flawed... assuming that the 1911 EB actually did say that homosexuality was a mental illness caused by eating too much cabbage, it would not be Original research to mention that in an article. The 1911 EB is being appropriately used as a secondary (or tertiary) source in that example. You could proabley challenge inclusion of the statement on reliability grounds (the 1911 EB being almost 100 years out of date), but not on NOR grounds. It simply isn't a WP:NOR violation, as no Original Research was involved. Blueboar (talk) 13:36, 9 January 2009 (UTC)

I believe you misunderstood. I am talking about inclusion of such a hypothetical statement about homosexuality in the article Encyclopædia Britannica Eleventh Edition, in which the text of the 1911 EB would be a primary source, as per your argument above. I feel that a quote like that should only be included in Encyclopædia Britannica Eleventh Edition if a RS had remarked upon it. In other words, Wikipedians should not go through the text of this edition and identify passages that they think are worth quoting in our article. Any quotes included on that basis, without support of a reliable, published secondary source, should be removed, regardless of their potential entertainment or amusement value. Would you agree? Jayen466 14:35, 9 January 2009 (UTC)

Oh, I see... Yes, in the article about the Encyclopædia Britannica Eleventh Edition, the 1911 EB would indeed be the primary source. However, we still would not be dealing with an NOR situation. In our hypothetical, the statement is a descriptive comment about what is (hypothetically) contained in the source, and is directly verifiable by the source. WP:NOR would not be grounds for removing the statement... there might be other grounds for removing it, but not NOR. Again, no one has introduced OR into the article.

I basically agree with your point about going through a source to find provocative or entertaining snippets to quote, but this has more to do with the relevance of the information and good article writing... I don't think it has anything to do with WP:NOR. Blueboar (talk) 14:58, 9 January 2009 (UTC)

Well, my point is that if I sit down with the 1911 EB and decide to look for "funny stuff", that is a form of original research: because it is the sort of research a scholar or writer analysing the work – say, to chronicle changing moral standards in the English-speaking world – might undertake. Except they wouldn't be doing it to score a point or raise a laugh, but with some legitimate educational purpose warranting publication of their work.

So, I still think selecting primary-source material to quote in an article is a form of OR, because it is a job that should be done by secondary sources; and if no reliable published secondary sources have done it, then we should not do it either. Happily, it doesn't necessarily matter whether we agree it is OR or not, as long as we agree that that is not how we should write articles. :-) As I said, I've also raised a similar concern at Wikipedia_talk:Neutral_point_of_view#Proposed_rewrite_of_WP:UNDUE. Jayen466 15:14, 9 January 2009 (UTC)

RE "Happily, it doesn't matter [w.r.t. WP:NOR] ...": Agreed-- happily it doesn't.
Kid in back seat, to Driver in front seat: "Are we there yet?"
Driver: "No"
Kid: "Are we there yet?"
Driver: "No"
Kid: "Are we there yet?
Driver: "Sorry to disillusion you, kid, but on this trip there is no 'there' " ... Kenosis (talk) 15:38, 9 January 2009 (UTC)

• To the first question, "what is newly forbidden by the primary sources section that was not already forbidden for all sources": it isn't an additional exclusion; it's elaborating, to the editor, that primary sources cannot provide but very limited material-descriptive- for the article. A primary source can't furnish the editor material for analysis, interpretation, conclusions, overall context or influence in the field or broader culture, evaluation, etc. Example, take the primary source, the Nag Hammadi's Gospel of Saint Thomas[14] and imagine writing an article about it without secondary sources. Beyond maybe identifying some of the individuals who are quoted in it, it's obvious almost nothing meaningful or informative could be written about it that doesn't require secondary sources.
• To the subsequent question, "given that sources are primary for some purposes and secondary for others, what is a claim that can be made when it is secondary but not when it is primary?" This is obvious example. Dr. Frankenstein is quoted in Frankenstein's Monster, "I have discovered the elixir of life", primary source. The article elixir of life cannot cite, "discovered by Dr. Frankenstein" to Frankenstein's Monster. However, if Science magazine writes, "the first to discover the elixir of life was Dr. Frankenstein", the claim is citable. This is an obvious example but the principle is the same when it comes to less obvious subjects. Professor marginalia (talk) 16:50, 9 January 2009 (UTC)

• Well, but even there, the issue seems to be one of attribution - that is, the elixir of life article could say "Dr. Frankenstein claims to have discovered it." So the issue is primarily that, in certain highly reliable sources, we're willing to take claims at face value and not require detailed attribution. But notably, in the case where we're citing Science, we're not making interpretive, synthetic, or analytic claims. So, again, two questions.

1. Is it accurate to say that the main difference between using primary and using secondary sources is that claims to primary sources have to be clearly attributed?
2. Is it accurate to say that, for any source, only descriptive claims can be made, not interpretive ones? Phil Sandifer (talk) 22:41, 9 January 2009 (UTC)

"Perhaps" to the first question (I need to think about this a bit more)... "no" to the second question. You can use secondary sources to support interpretive claims about a topic.

• I was involved in a dispute over a claim not unlike the Frankenstein example. An editor had cited a book, Two Old Women, as an accurate account of an real episode in oral history put in writing. (Yes, I know :-), but the other editor argued it was a cultural bias to call the story fiction, and works of fiction don't always announce they're not pretending to be true, do they?) It was somewhat like claiming a 7th century storybook about St. Patrick was a primary source for, "and that's why today there are no snakes to be found in Ireland to this day." So while the book may say "that's why there are no snakes", is it sufficient simply to attribute it as a primary sourced claim? (Another confusing dispute I found myself in ran the other way, when an editor kept misrepresenting a work of non-fiction as a fictional novel. Takes all kinds.) I'm not trying to say that these weird examples illustrate how to write any specific rule. I think they're sometimes useful to help us see where some of the outlines of the problem might be. I think what hasn't been spelled out well is that we usually are using judgment about how to read primary sources. We don't expect from Frankenstein the same kind descriptive information we do from Autobiography of Benjamin Franklin, or the same kind of description of Unibomber manifesto as the Unibomber trial transcript. Maybe that's one reason it's difficult to nail this tight.
• Does it make the descriptive/interpretative distinction easier if it's thought of as two kinds of claims that can be made about a subject rather than two kinds of claims that can we can make from about a source? Professor marginalia (talk) 01:26, 10 January 2009 (UTC)

I certainly think it makes the distinction clearer. We can cite a Primary source for a descriptive statement about a subject, but if we want to make an interpretive statement about the subject, we need a secondary source.

We will still get occasional confusion when the subject happens to be a text... because that text is also a source (the primary source for that subject). But I think if we keep the concepts of "Text-discussed-as-subject", and "Text-used-as-source" separate in our minds as we write, then we can see the distinction clearer.Blueboar (talk) 05:16, 10 January 2009 (UTC)

I think that's a really useful distinction, actually. Can we clarify it in the policy? Phil Sandifer (talk) 16:13, 10 January 2009 (UTC)

I don't have an objection to that... Perhaps changing from:

• "Any interpretation of primary source material requires a reliable secondary source for that interpretation. Without a secondary source, a primary source may be used only to make descriptive claims, the accuracy of which is verifiable by a reasonable, educated person without specialist knowledge."

to:

• "Any interpretation of a subject requires a reliable secondary source for that interpretation. Primary source should be used only to make descriptive claims about the subject, the accuracy of such claims should be verifiable by a reasonable, educated person without specialist knowledge. Blueboar (talk) 17:59, 10 January 2009 (UTC)

• That is a *lot* better than what we had before I started pissing and moaning. :) Phil Sandifer (talk) 18:14, 10 January 2009 (UTC)

OK... if we are both in agreement, it is probably good language... but just to make sure, I will post it as a "proposed change" (I think it may get lost at the end of this long exchange, and I want to make sure others see it and have a chance to comment) Blueboar (talk) 23:17, 10 January 2009 (UTC)

Proposed criteria to replace the "directly related" test

The policy states:

Citing sources and avoiding original research are inextricably linked: to demonstrate that you are not presenting original research, you must cite reliable sources that are directly related to the topic of the article, and that directly support the information as it is presented.

Problem: "Directly related" is too vague. Solution: Propose specific criteria in replacement of the "directly related" test.

Proofs: (logical extension of the WP:SYN rule to implied and suggested conclusions)

Example 1 - Deduction: (Smith & Jones example - article is about "Jones")

• A = Smith says that Jones committed plagiarism by copying references from another book.[properly referenced]
• B = The Harvard manual does not call violating this rule "plagiarism".[source does not comment on dispute] (relevant information)
• C = Therefore, if Jones copied references from another book, he did not commit plagiarism according to the Harvard manual.[no source, or source does not comment on dispute]

"A and B, therefore C" is original research ("original synthesis")

Example 2 - Deduction: (Modified S&J example)

• A = Smith says that Jones committed plagiarism by copying references from another book.[properly referenced]
• B = The Harvard manual does not call violating this rule "plagiarism".[source does not comment on dispute] (relevant information)
• [Implied conclusion: C = Therefore, if Jones copied references from another book, he did not commit plagiarism according to the Harvard manual.]

"A and B" is original research because "A and B" implies "A and B, therefore C".

Criterion (A):

• Statements verified by sources which do not refer to the subject-matter of the article are deemed to constitute original research when such statements clearly imply and logically entail non sourced, noteworthy conclusions about a "topic" related to the article subject.

Example 3 - "Suggested" conclusion: (Paleolithic diet article)

• A = The Paleolithic diet excludes grains, including whole grains, on the grounds that they promote ill-health.[properly referenced]
• B = Consumption of whole grains has been associated with a decreased risk of several health problems.[source does not refer to the Paleolithic diet, nor does it comment on key related topics such as the Paleolithic, human evolution, hunter-gatherers or evolutionary medicine] (relevant information)
• C = Exclusion of whole grains from the Paleolithic diet may not be justified. [no source, or source does not refer to the above key related topics]

"A and B, therefore C" is original research ("original synthesis")

Example 4 - "Suggested" conclusion: (modified paleodiet example)

• A = The Paleolithic diet excludes grains, including whole grains, on the grounds that they promote ill-health.[properly referenced]
• B = Consumption of whole grains has been associated with a decreased risk of several health problems.[source does not refer to the Paleolithic diet, nor does it comment on key related topics such as the Paleolithic, human evolution, hunter-gatherers or evolutionary medicine] (relevant information)
• [Suggested conclusion: C = Exclusion of whole grains from the Paleolithic diet may not be justified.]

"A and B" is original research because "A and B" suggests "A and B, therefore C".

Criterion (B):

• Statements verified by sources which do not refer to the subject-matter of the article are deemed to constitute original research when such statements evidently suggest, but do not logically entail, non sourced favorable or unfavorable assessments about a "topic" related to the article subject.

Criterion (C):

• Editors should build an article by summarizing the sources available on the topic of the article. Any information added should therefore be based on reliable sources that present this information in direct connection with the article subject. In some cases, supplementary information from generic sources that cover a broader subject area than the specific article topic may be deemed to add value to an article, in order to clarify places, people or things mentioned in the article (WP:PCR). For example, an editor might want to add a detail from a reliable source that describes the historical context in which the subject of an article lived, even though the cited source does not mention the article subject. This is fine as long as it does not entail a substantial new conclusion about the article subject that is not present in any of the sources.

Note: This is just a draft proposal, so the specific wording of the criteria can be improved and will need to be integrated. The proposal does not affect the requirement to cite reliable sources that directly support the information as it is presented, not does it affect the WP:SYN rule. Let's see where this goes.

--Phenylalanine (talk) 05:53, 9 January 2009 (UTC)

Perhaps add at the end, "... does not mention the article subject. This is fine as long as it does not entail a substantial new conclusion about the article subject that is not present in any of the sources" or some such wording. Otherwise, I agree with the examples and your analysis of them. Jayen466 10:52, 9 January 2009 (UTC)

Done. Thanks for the suggestion. Cheers, Phenylalanine (talk) 02:10, 10 January 2009 (UTC)

Some of the headings that say "Criteria" only discuss a single criterion. Am I missing something (quite possible)?IceCreamEmpress (talk) 01:52, 10 January 2009 (UTC)

Thanks for pointing out the mistake. Fixed. Cheers, Phenylalanine (talk) 02:10, 10 January 2009 (UTC)

Here's an essay I created on this subject: User:Phenylalanine/Synthesis of published material which advances an implicit position. Comments and suggestions are welcome! Phenylalanine (talk) 04:09, 11 January 2009 (UTC)

Question about definition of a primary source

Apologies for asking something that I could in principle figure out, but this article changes so rapidly that it might take me hours: when did the list of types of primary sources lose "experimental results written by the person(s) who conducted the experiments"? That's really critical for articles about scientific or medical topics -- otherwise it becomes extremely difficult to defend against "cherry picking". If this has been discussed -- as I expect it has been -- a pointer to the discussion would be sufficient for me. Thanks, Looie496 (talk) 18:40, 10 January 2009 (UTC)

I'm going to replace it right now. In general, for purposes of WP, the first published report of a particular study is a primary source. Unlike some fields of scholarship, WP is simply not properly in the business of going through researchers' lab notes, field notes, etc. ... Kenosis (talk) 21:13, 10 January 2009 (UTC)

Looks like this is going to be a battle also. FWIW, I agree it's good example of PS to list. Every week Science publishes research, often research that conflicts with earlier research published in the same journal. One recent example is a study concluding that a residue of nanodiamonds in 13,000 yo strata worldwide is evidence of catastrophic comet collisions which led to the extinction of the Clovis and large mammals. This is a solidly researched, peer reviewed, finding that's far from definitive. This primary sourced claim can't be described in a way it's given more authority than secondarily sourced claims representing a more widely accepted alternative hypothesis. Extra caution is most definitely required for citations to primary sourced research. Professor marginalia (talk) 19:30, 11 January 2009 (UTC)

Proposed Change to PSTS

Per the discussions above... I propose changing the language of WP:PSTS (primary sources) from:

• Any interpretation of primary source material requires a reliable secondary source for that interpretation. Without a secondary source, a primary source may be used only to make descriptive claims, the accuracy of which is verifiable by a reasonable, educated person without specialist knowledge."

to:

• "Any interpretative statement about a subject requires a reliable secondary source for that interpretation. Primary sources should be used only to make descriptive statements about the subject, the accuracy of which should be verifiable by a reasonable, educated person without specialist knowledge.

Please comment. Blueboar (talk) 23:23, 10 January 2009 (UTC)

It does not match my understanding. If I use a primary source to make a descriptive statement about a subject, then I use the primary source as a secondary source. I have always understood "descriptive claims" based on a primary source to be descriptions of that primary source itself. E.g., "Jane Doe says in her autobiography that she never enjoyed being a singer"; "John Doe says on his website that he was born in 1964." These are descriptive claims about primary sources. In the proposed wording, these would become descriptive statements about the subject. Obviously, if we take them as statements about the subject rather than as descriptions of primary sources, then we cannot verify their accuracy: so they are not "verifiable by a reasonable, educated person without specialist knowledge". The old wording works better for me. Jayen466 04:37, 11 January 2009 (UTC)

See, I'm actually increasingly troubled not only by the treatment of primary sources by that restriction, but by the treatment of secondary sources, as that implies that secondary sources *can* be used for interpretive, synthetic, analytic, etc claims. Phil Sandifer (talk) 05:37, 11 January 2009 (UTC)

TBH, I'm OK with the language either way. This is one of those subtleties that present themselves in this editorial policy, which must work across an extremely wide range of topic areas on the wiki. The other two content policies each has its rough equivalent in terms of being difficult if not impossible to capture the exact language that will work across the whole wiki. In WP:NPOV, the most difficult quirks tend to revolve around distinguishing content forks about a particular point of view from POV forks, and in WP:WEIGHT, where the notion of "in proportion to the prominence of each [viewpoint]" is frequently meaningless when dealing with very prominent unreliable rumors, written gossip, "urban myth" and such. In order to reconcile this, we must cross over into another policy and apply, e.g., WP:V#Reliable_sources. In WP:V, we see similar problems in finding language that can apply across the whole wiki, particularly w.r.t. around RS and the always quirksome issue of distinguishing between verified and "verifiable"-- e.g., whether all uncited information should be deleted. The policy says it may be deleted, though the reality is that that doesn't work in every case. So the language, although long accepted, never quite captures the reality in every case across the whole wiki.
...... In each case in the other two content policies, we're expected to apply the consensus process to reconcile disagreements about how to apply the policy in a given circumstance. Same thing here. In general, the language works just fine in the overwhelming majority of instances across the wiki where the consensus process is applied in good faith. ... Kenosis (talk) 06:00, 11 January 2009 (UTC)

Hmmm... Jayen, I see what you are saying... But I think both of your examples are simply an attributed way of making a claim about the subject. The "claim" is that the subject said these remarks (the website or autobio are simply the locations of the remarks). What the policy is telling us is that we can report that the subject made the remarks, but can not draw conclusions from these remarks. For that we would need to rely on secondary source. Blueboar (talk) 19:17, 11 January 2009 (UTC)

It occurs to me that my own example above about the nanodiamonds and mammoth extinction could result in confused interpretations about what even "the subject" really means. The study gathers data, analyzes certain evidence and offers it up as a chink linking all these different hypotheses about comet collisions, nanodiamonds, global climate change, large mammal extinctions, and the sudden disappearance of a pre-columbian culture. It didn't really contribute any primary research in terms of findings on "the subject" that there actually were these large mammal extinctions, for example. Professor marginalia (talk) 20:35, 11 January 2009 (UTC)

How to deal with published, refereed experiment reports?

I just added a paragraph to the page stating:

The classification of "experimental results written by the person(s) who conducted the experiments" (which include the majority of scientific and medical publications) as sources to be avoided is sometimes seen as puzzling, especially by scientists who are taught that these are the best possible sources. The reason for being careful with them is that it is common for important questions, especially ones relating to medicine, to be investigated by large numbers of experiments, even hundreds in some cases. It is very difficult to summarize the findings of such a large number of publications without engaging in original synthesis, sometimes known as cherry picking. Therefore Wikipedia policy strongly prefers making use of secondary sources such as review papers, perspective articles, or editorials in mainstream medical journals, whenever possible.

Phil Sandifer then reverted this, and also removed experimental results written by the person(s) who conducted the experiments from the list of primary source types, using an edit summary stating, I have real trouble with NOR actively cautioning against such a large swath of reliable, peer-reviewed sources. I believe I explained why this is necessary: can we discuss the issue? Looie496 (talk) 18:48, 11 January 2009 (UTC)

I think the problem may be with the parenthetical remark... It could be taken to mean that we should not cite the majority of scientific and medical publications. I am sure that was not your intent, but that is how it reads to a non-scientist. Blueboar (talk) 19:23, 11 January 2009 (UTC)

I think once we start getting too specific about particular domains, science, literature, etc., guidelines are better. I think it's sufficient here to include published scientific research papers as one example of primary research. I do like the way it's pointed out that it's often impossible to summarize a body of primary research without synthesis, which is OR here. My brain's getting numb from all the policy talk, though. Maybe this is already clear in the policy. -¿º Professor marginalia (talk) 19:44, 11 January 2009 (UTC)

I flat-out oppose anything that suggests that editors should avoid peer-reviewed sources. In any field. Period. I think it's an appalling idea that serves to needlessly gut our coverage in numerous areas for no practical gain. The creep of OR from "avoid novel synthesis from primary sources" to the bloated hydra it has become where any sort of claim that is not present all but word for word in the cited source is OR needs to not only stop, but be reversed. This addition, attempting to bar sources that are as reliable as they come because God forbid, they might require actual thought to use is exactly what this policy does not need. Phil Sandifer (talk) 20:07, 11 January 2009 (UTC)

Phil, I agree with your reversion, though for different reasons than you give here. Thank you for making your basic position more explicit than you had made it before. ... Kenosis (talk) 20:45, 11 January 2009 (UTC)

Phil, I think you forget you are on the WP:NOR page, not the WP:V page. A peer-reviewed scientific primary research paper is reliable insofar as what it presents and to the audience it is written. It's use on WP is very, very limited, mostly for NOR issues. I disagree with "avoid", but once WP editors realise just how limited such a source is, they will prefer to find better ones. The scientific and medical world is amply served by secondary sources (for example, a literature review in a medical journal) that are much more useful as sources on WP. The idea that all peer-reviewed literature is "as reliable as they come" and therefore we should never caution against their use is naive. It is much more complex than just "reliability". Colin°^Talk 20:55, 11 January 2009 (UTC)

RE User:Looie496's offering above: For peer-reviewed sources follow the same rules as for any other source, WP:V#Reliable_sources and WP:RS analyses, as well as WP:WEIGHT and any other policies that apply. WP:NPOV directs us to give the reader a neutral "perspective" of what the reliable sources say about a topic. Avoiding "cherry picking" is part of doing this, which is why this policy says: "Primary sources that have been reliably published (for example, by a university press or mainstream newspaper) may be used in Wikipedia, but only with care, because it is easy to misuse them. ..." WP:PSTS proceeds to state in somewhat more specific language the limits on the use of primary sources that we're expected to observe. ... Kenosis (talk) 20:36, 11 January 2009 (UTC)

Agreed. It shouldn't be, "editors should avoid peer-reviewed sources" but, "editors must use primary sources cautiously", as in "avoiding novel synthesis of primary sources", or "avoiding original analysis of the primary-sources." And please let's not trial balloon new wording on the policy page anymore, okay? We need consensus on the talk page first for anything but the most minor changes. Professor marginalia (talk) 20:52, 11 January 2009 (UTC)

I agree with much of what Looi496 wrote (except the word "avoid" is too absolute and also except that editorials are opinion pieces, so not really suitable as sources other than for the opinion of the journal's editors, which usually isn't notable). The vast majority of published scientific and medical articles are not suitable sources for an encyclopaedia with a NOR policy. However, I don't believe the text belongs in general policy. See WP:MEDRS for the appropriate guideline. Two issues that do belong here, which Looi496 touches on are:

1. Researchers are trained to read and cite primary scientific literature. We are not researchers. To avoid OR, it is preferable to cite expert analysis of the primary literature.
2. The definition of primary sources needs to include primary scientific or medical research papers. The absence of this is causing confusion. The definitions cited by this policy already include them, as do the more scientific/medical definitions I listed here.

Our PSTS section is still very poor IMO. The "key point" quoted in this policy is rubbish ("insider's guide" -- what does that mean). The key point of primary sources is that they are original materials or original thought that has not been previously published in any form in any other source. I shall have a think about some improved wording. Colin°^Talk 20:58, 11 January 2009 (UTC)

The "key point" thing you refer to was added within the last year (anybody recall or care to dig through and find out when?). I'm not sure that saying "the key point about primary source is that it provides an 'insider's guide' ... " is the best way of summing it up for most readers of the policy page. Nor do I have any particularly vehement objections to it. ... Kenosis (talk) 21:48, 11 January 2009 (UTC)

I tend to think that the issue is not with primary scientific literature, but with advanced scientific literature. Some subjects are hard. Not everybody can write about them well. "It's hard" is not a valid reason to avoid covering it, though. Phil Sandifer (talk) 21:44, 11 January 2009 (UTC)

That may be true... but, "it's hard" isn't a valid excuse for inserting OR either. Whether the topic is easy or hard, Wikipedia's goal is to report what others say about it, and not to introduce our own ideas about it. Blueboar (talk) 22:17, 11 January 2009 (UTC)

The collapsing of "encyclopedic knowledge" to "the knowledge contained in secondary sources" is an unfortunate fallacy that does us no good. Our task is to report significant viewpoints. Those published in peer-reviewed journals have already cleared that threshold. Phil Sandifer (talk) 01:06, 12 January 2009 (UTC)

You are, once again, quite apparently confusing the process of research -- more specifically writing about original research and adding one's own original research or synthesis in such writing, as is [typically] required for, say, a dissertation -- with on the other hand writing or editing an encyclopedia article. ... Kenosis (talk) 05:44, 12 January 2009 (UTC)

It would really help if the reliability of a publication (for example, that it has been peer-reviewed) was totally divorced from issues of NOR in the use of that publication for writing encyclopaedia articles. All hard subjects, of any notability, are covered by secondary sources. This is not a question of "not covering" certain topics. It is about what sources to use when covering topics, hard or otherwise. Colin°^Talk 22:45, 11 January 2009 (UTC)

First of all, it is not as though Wikipedia limits itself, or is even primarily focused on "hard subjects." Second of all, while I will grant that coverage by secondary sources is a good sign of notability, it in no way follows from that fact that the whole, or even the bulk of what is worth saying about a topic has been covered in secondary sources.

This idea that anything that has not appeared in a secondary source is original research is nonsense. Phil Sandifer (talk) 01:06, 12 January 2009 (UTC)

(Undent) WT:MEDRS debated this issue for several tedious weeks last year, and we might be able to save some time if people here would go off to the archives and read the discussions and the outcome.

Here's a short summary: Medicine-related articles are an important subset of Wikipedia articles that are affected by WP:PSTS. We (meaning all reasonable and thoughtful editors, which I assume includes every person reading this page) want to have good sources. We don't want some strange outlier to distort our articles. We want to end up with good, solid, reliable information.

The best way to get this information is to deal with solid secondary sources. You let a reputable expert (or several of them) look over all the stuff and sum it up, and you work from his or her papers. (If published in scientific journals, these are reviews, which encompass systematic reviews to answer narrow points using a stated methodology, and literature reviews or narrative reviews, which is a broader, more descriptive approach.) This is fantastic, and it neatly solves both obvious SYNTH (paper X plus paper Y equals conclusion Ω) and also the DUE problem associated with overemphasizing an editor's 'favorite' primary paper. This is good advice even if the primary paper was published in a peer-reviewed journal: Good peer-reviewed journals publish mistakes all the time, and lousy journals do it as a matter of routine. This is good advice even if the Wikipedia editor is a real-world expert: Real-world experts make mistakes all the time. This is good advice for nearly every situation, and we all want to encourage it.

However: solid secondary sources are not always available for every single relevant aspect of every single (bona fide/WP:N-meeting) medical condition. So the exceedingly rare Oculodentodigital syndrome, which is my favorite example of this problem, deserves space in Wikipedia, but there aren't any solid secondary sources. The best source is actually a case study that also happens to review the condition (presumably so the readers would have some clue what the author was talking about when he described his specific cases). In many more cases, a good review will exist, but we have to turn to primary literature to add information about newer treatments, current epidemiological information, or other details. In short, we cannot assume that solid secondary sources will always be sufficient. This is why WP:MEDRS says that medicine-related articles are ideally based upon high-quality secondary sources (reviews and medical textbooks), not limited to these sources.

What matters is not whether you ever use a primary paper; what matters is how you use it. (See, for example, WP:MEDRS#Respect_secondary_sources.) I see no reason why this page can't (1) identify many examples of primary sources (including the primary scientific literature) and (2) caution against their inappropriate use (with, if desired, many examples) without unduly constraining good editors. We do not want to ban primary scientific literature, but we also do not want to permit its abuse, either. WhatamIdoing (talk) 02:13, 12 January 2009 (UTC)

Well said, WhatamIdoing. Thanks for reiterating this perspective with such an effective summary of the issues. As to your last sentence, nobody on the wiki is constrained by any version of WP:PSTS that has ever been here since it started a couple years ago, except those who attempt to use primary sources in an inappropriate way. Those who use primary sources with the extra caution described in the policy and within the limits stated in WP:PSTS are by no means at any risk of suffering even the slightest degradation of their efforts to contribute encyclopedic material to WP. ... Kenosis (talk) 03:29, 12 January 2009 (UTC)

• Research papers are quite inaccurate and being peer-reviewed in a respectable journal is not nearly enough to offset this. For example, see Publish and be wrong which says, "...a study of 49 papers in leading journals that had been cited by more than 1,000 other scientists. They were, in other words, well-regarded research. But he found that, within only a few years, almost a third of the papers had been refuted by other studies." I have reverted in support of this position that we should be suspicious of such material. Colonel Warden (talk) 10:23, 12 January 2009 (UTC)

The essay in the Economist oversimplifies the issue and is basically wrong. But then, you'd expect that from economists. See the original paper, which is quite readable. 1,000 citations is huge. These are papers in areas undergoing very active research and considerable controversy. Such studies beg further research. The Facts: of the 49 papers they picked, then then selected just 45 that had positive results. Of those 45, 7 were contradicted by subsequent research (16%). Another 7 had the strength of positive effect reduced by subsequent research (they weren't contradicted, we just learned more). So 84% of these papers produced results that were essentially right as far as we know today. What is interesting is that the paper drew on meta-analyses to indicate the current scientific status on the research topic (ie. to show whether what we know now contradicts what they thought then). These are secondary sources. We should do likewise. Assuming that a single primary research paper produced results that reflect "current scientific opinion" is original research. BTW: I still disagree with the inclusion of the text in WP:NOR. Colin°^Talk 13:11, 12 January 2009 (UTC)

Of course, if we were to take an anti-journal article stance, we would be forced to rely on summaries from the Economist and other mass market news sources over what the articles actually say. Which, given the well-documented legacy of bad descriptions of science even in good papers like the NYT and the Economist, is an unsatisfying proposition to say the least.

Best, I think, to stick with the best available sources. And you don't beat peer-reviewed journals. In any field. If peer-reviewed sources are available, they are preferable in almost every case. (I'll grant an exception for peer-reviewed scholarship of popular culture, where basic historical facts about are probably better sourced elsewhere if possible). Phil Sandifer (talk) 15:35, 12 January 2009 (UTC)

This proposal seems to be a conflation of WP:NOR with WP:DUE. I agree with Phil Sandifer's reversion, as any editor with a basic scientific education can eventually figure out what a peer-reviewed article abstract is saying to summarize its own experimental results, without engaging in any original research or documentary/historic analysis. --Hroðulf (or Hrothulf) (Talk) 16:10, 12 January 2009 (UTC)

I'm worried about the new language here, because it could, I think, be wrongly taken to mean that journal articles are published experimental results. Experimental results, I think, especially in the context of the rest of PSTS, is meant to be the raw data - not the heavily analyzed, peer-reviewed, and multiply authored results of a scientific journal publication.

I do not think this change makes sense, or fits with consensus. Phil Sandifer (talk) 17:05, 12 January 2009 (UTC)

Are peer-reviewed sources primary?

I mean this as a sincere question. As I look at the primary sources section, it seems to stress the immediacy of the primary source - which is what makes it so difficult for us to use. Basically, the primary source is uninterpreted data - whether that data be experimental results, historical documents, or literary.

Nowhere in the text does it say anything that comes close to a peer-reviewed, academic source being a primary source - and that makes sense to me. In the sciences, there's a huge line between lab notebooks (what are banned by PSTS) and published results, which tend to have A) repeated experiments, B) analysis of data, C) multiple authorship, D) peer review, and E) editing in response to peer review.

In the humanities, some of this disappears - but I know of nobody who publishes without getting extensive comments from their colleagues, and then peer review and editing still take place. So we're still, with academic sources, dealing with stuff that is very, very far from immediate. And academic publishing is not about presenting an insider's view of events - it's about summarizing research, explaining conclusions, and presenting things so that they are known and referencable. Which is very, very different from primary sources.

I'm wondering if we should not expressly say that peer-reviewed sources are not primary. Phil Sandifer (talk) 16:04, 12 January 2009 (UTC)

Isn't the major source of trouble source typing itself? What you've suggested has been discussed before, surely more than once. I think Blueboar is on the right track when he says the real problem is misuse of sources, not the type of source misused. We are shown examples of how editors have misused primary sources. I think Blueboar's position is that it is the misuse that is bad, and that would hold no matter what type of source was misused. Also, logically, showing that primary sources are sometimes abused does not provide grounds for forbidding the use of primary sources. Secondary sources surely are also misused. I recognize that such use is not forbidden: only careless use of primary sources is forbidden. To me that would mean that an editor is wrong to remove any material because (and only because) it is based on a primary source. What the editor should demonstrate is that the source was used carelessly. Which is pretty much the same thing Blueboar appears to advocate: don't misuse sources. If the editor can show that the source was used carelessly (or simply improperly) it doesn't matter whether the source was primary (by either accepted or Wikipedia-specific definition), secondary, or tertiary. The matter based on careless/improper use of a source should be removed.

I am far from the first to suggest that source typing be expunged from Wikipedia:NOR. I still think it is not just a good idea but is the best idea. Let the focus be on the proper use of sources. What useful purpose is served by Wikipedia having strained and idiosyncratic definitions of "primary" and "secondary"?

I was taught (as a chemist) that for experimental articles the experimental data in a journal article is pretty generally reliable and that the interpretations are less so. Minasbeede (talk) 16:21, 12 January 2009 (UTC)

I am open to the idea that PSTS does not have consensus, or that it is, at best, guideline-level consensus. It has always been the most controversial part of this policy. Phil Sandifer (talk) 16:25, 12 January 2009 (UTC)

(edit conflict) Absolutely. A book by Richard Dawkins (or Carl Sagan) is less reliable than a peer-reviewed article by the same man, since it has been checked by fewer people, even though it comes from a major publisher and hits a bestseller list. Of course, either could be out of date and superseded by better theories or more accurate experimental results. --Hroðulf (or Hrothulf) (Talk) 16:26, 12 January 2009 (UTC)

Well, let's bracket academic press publication, which amounts to peer reviewed books. :) Phil Sandifer (talk) 16:42, 12 January 2009 (UTC)

Phil, you may be sincere but this is really misguided. Please find a source for your observation that "the primary source is uninterpreted data". All the sources I have found (e.g. this) group both lab books and the initial publication of results as primary sources. To quote:

"Primary sources present information that has not been previously published in any form in any other source. These sources may evolve through either formal or informal channels of communication. Journals, patents, and technical reports are examples of primary literature that have been evaluated through a peer-review process and are disseminated through published sources. Other primary sources, such as laboratory notebooks, memoranda, e-mail or listservs, are not usually published, but are nevertheless an important resource."

The levels of edtiorial and peer review that a work of literature has undergone has nothing to do with PSTS. It certainly affects reliability, but that's another story. Colin°^Talk 17:06, 12 January 2009 (UTC)

Well, literature doesn't go through peer review as such, and I'd argue that the editorial review is substantially different once you introduce the idea of artistic vision. Phil Sandifer (talk) 17:12, 12 January 2009 (UTC)

I'm using the word "literature" to mean published writing on a subject, rather than specifically artistic literature. The "scientific literature", for example, is often peer-reviewed. But this remains a red herring wrt NOR. Colin°^Talk 17:28, 12 January 2009 (UTC)

"less reliable" sort of implies that Dawkins and/or Sagan (as examples) were ineffectual in creatng and reviewing their own work to make sure it is valid. Either of those authors, I'd strongly suspect, would be able to distinguish between solid material that they take from another source and less solid material - material that amounts to a being a new, untested hypothesis. I'd also suspect the two could make that distinction even if they were in agreement with the hypothesis: they'd recognize the nature of the process needed to bring the hypothesis from untested to established (or possibly to rejected.) In dealing with the content of such a book the task of the Wikipedia editor is to make a similar distinction and to avoid using material that is has not been generally accepted as though it has been generally accepted. In any event, the emphasis isn't on "reliable" as far as this discussion is concerned (which may be bad or good: I'm just repeating something I read in the discussion.) Conceivably there could be a passage in such a book which has content that has been fully peer reviewed and is generally accepted - but still might be misused by a Wikipedia editor. (In other words I'm lobbying for the concept that it is misuse that creates the problem, not the type of source used.) If the problem is misuse then there is no amount of effort that can be put into rigorous definition (even if idiosyncratic) of what "primary" means that will cure the problem. The definition effort is misguided: it is misuse (even if unintentional) that corrupts Wikipedia, not the type of source cited. Minasbeede (talk) 17:14, 12 January 2009 (UTC)

Minasbeede, I agree that proper use of any source is more important than a classification of the source. I would be happy if WP:PSTS were removed from WP:NOR and demoted to a guideline or essay. --Hroðulf (or Hrothulf) (Talk) 17:37, 12 January 2009 (UTC)

Hold on... having been through several debates that proposed both removing PSTS and demoting it to a guideline, I know there is a strong consensus against doing either. I can all but guarentee that any attempt to remove it will be met with adament objections.

I think a far more productive discussion is whether we should shift the focus of the section from "Because people often misuse Primary sources, we have determined that you can only use primary sources to say X" (which causes a prolbem because we can not agree on what X is, not to mention agreeing on what exaclty falls into the category of Primary source) to "Primary sources are easy to misuse, so be very very careful... here are several examples of ways primary sources can be misused... and here are some ways they can be used appropriately."

Keep the focus on the broad principle of not adding original research. Blueboar (talk) 18:20, 12 January 2009 (UTC)

Whether or not it is "consensus" I am aware that there is strong advocacy of source typing. At one time I think that Wikipedia:Consensus warned of a situation in which over time the same objection is raised and fought off by a faction, with the result that those objecting give up, to be later replaced by others making the same objection.

It seems that (a) source typing is particularly favored by a group of editors whose focus is on social sciences and humanities and that (b) those in physical sciences are frequently disturbed by source typing, which appears to fly in the face of what physical scientists do and find reasonable. The source typing advocacy has a narrow focus (social sciences) but the advocates wish to make it a general policy, across all of Wikipedia. It also seems to be the case that source typing ought to be in Wikipedia:verifiability but doesn't really work there, so it is instead put into Wikipedia:NOR, where it resides uncomfortably.

In addition Wikipedia:NOR forbids original research, which is fine and proper. If a use of a source amounts to original research then the material needs to be altered or removed. That seems to be the major point. How the source material is classified is of far less importance than the fact that it is misused. Further, the actual use of the source typing clause in Wikipedia:NOR appears to be as a reason to arbitrarily (without discussion) remove material. The policy says to use primary sources with care but the actual usage is consistent with an interpretation that says don't use primary sources at all.

I agree: "keep the focus on the broad principle of not adding original research." I think source typing removes that focus. My involvement in this discussion started in November, 2007 (or before.) Back then there was discussion of what was and wasn't a "primary" source. Today the same discussion continues. That seems to me to be a sign that source typing is too spongy in its definition to ever fit in a policy.

I fully agree that source typing advocates were adamant and likely will continue to be so. I take comfort in the notion that most Wikipedia editors do a good job and are actually unaware of this long-raging conflict. That is, however this discussion goes and no matter how long it goes on most editors will do things which enhance Wikipedia, with this discussion (and source typing) not mattering much at all.

In any event this has come a long way from being a means of preventing the advocacy of screwball theories. I'm not convinced that all the additions to Wikipedia:NOR have effected an improvement in it. Minasbeede (talk) 18:49, 12 January 2009 (UTC)

"Primary source" and "peer reviewed source" are two separate characterizations that may or may not overlap with a particular source. Why is it that its "peer review" now getting dragged into the PSTS discussion? What has peer review to do with the original research policy? Seriously! Professor marginalia (talk) 18:23, 12 January 2009 (UTC)

I think the concern is that by adding "experimental results written by the person(s) who conducted the experiments" we are somehow "banning" peer reviewed journals (because such material appears in peer reviewed journals). I think this is a good time to remind people that this policy (even in it's current state) does not BAN the use of any primary source... it limits how we use them, but not their use. Blueboar (talk) 19:51, 12 January 2009 (UTC)

(ec)I see. Then my short, sweet answer to the question, "If we should not expressly say that peer-reviewed sources are not primary?" is absolutely not. Thanks. Professor marginalia (talk) 20:08, 12 January 2009 (UTC)

Something I am still fuzzy about is exactly what is permitted of the sources that NOR calls secondary that is not permitted of the sources that NOR calls primary. — Carl (CBM · talk) 19:57, 12 January 2009 (UTC)

That is actually quite easy... Statements that analyse or interpret something, or reach a conclusion about something must be backed by secondary sources. You should not base such statements on primary sources. Blueboar (talk) 21:21, 12 January 2009 (UTC)

That's not what NOR says, though. It says that "Any interpretation of primary source material requires a reliable secondary source for that interpretation." Interpretations of secondary source material do not (apparently) have this requirement. For the record, if NOR clearly said what you just said I would have no strong objection to it. — Carl (CBM · talk) 21:49, 12 January 2009 (UTC)

I think we are all clear that if the so-called primary source is itself the subject of the article text (for example, the novel) then you need a secondary source on the novel in order to include analysis of the novel on WP. This is why we say "All interpretive claims, analyses, or synthetic claims about primary sources must be referenced to a secondary source". Note that this is "must" not "is preferably". But if the primary source is merely close to the subject (a diary of a person, a report of a scientific study) the need for a secondary source becomes just strong advice rather than a requirement. Far better to cite the biography of the person, or a literature-review of the scientific topic, than to attempt to become a biographer or reviewer oneself. That path leads to original research... Colin°^Talk 21:34, 12 January 2009 (UTC)

Yes, in general terms I agree.

One thing that I have noticed is that our standard practice is to use mainly primary sources when writing biographical articles about prominent people. For example George W. Bush and other articles on current politicians are written mostly from newspaper articles. So the idea that we should not be biographers ourselves, while it has some merit, is only followed when there is a sufficient mass of historical research that we can rely on secondary sources. As a though experiment, imagine if our article on Mahatma Ghandi was written so that more than half the citations were to newspaper articles written during his lifetime. — Carl (CBM · talk) 02:10, 13 January 2009 (UTC)

RFC-Specialist knowledge

This discussion has been closed. Please do not modify it.
The following discussion has been closed. Please do not modify it.
RFC closed. Present policy reads, "a primary source may be used only to make descriptive claims, the accuracy of which is verifiable by any reasonable, educated person without specialist knowledge." Does this restriction unduly limit the use of primary sources? Does this restriction conflict with WP:NPOV by placing a higher restriction on primary sources than secondary or tertiary sources? There are degrees of specialization, and degrees of interpretation. No source is either totally reliable or unreliable, and all sources whatsoever require interpretation. It is not possible to write an encyclopedia without the use of intelligence, human or artificial--the necessary quality of judgment can not be bypassed. the use of intelligence in evaluating material is inherently research, and even when selecting such obviously tertiary sources as standard textbooks, we need to use intelligence and research in determining which ones to use and what parts to follow. The distinction between primary and secondary sources has proven really confusing here, because people in different subjects use it differently. To an historian, a newspaper is a primary source; to us, it is a secondary source. To a scientist, what we call a "primary journal article" is a mix of description and interpretation, and does not fit into the simplistic pattern used here. I am not at the moment sure of a better wording, but the present one is excessively rigid. DGG (talk) 06:34, 3 January 2009 (UTC) Whew. I can see the confusion with the distinction as I've run into it before elsewhere. Again we may need to use examples. And, sorry that I seem to be on a hobby horse at the moment, on the question of is this unduly restrictive, I'd need to know a bit more about why you think this might be the case, and I often work best with practical examples. dougweller (talk) 07:09, 3 January 2009 (UTC) I think DGG is onto something. Even with secondary sources, some amount of interpretation is necessary. People summarize. They clean up language. They highlight stuff that's important, and remove stuff that isn't important. When is it point of view pushing? You'll know it when you see it. I don't think the test is a bad one: something that can be verified by any reasonable, educated person without specialist knowledge. But I might also add that "if the interpretation of the primary source is disputed, then it's likely original research." After all, if there is more than one interpretation, then it's no longer a fact. Randomran (talk) 07:20, 3 January 2009 (UTC) Like Dougweller says, we need examples. Per WP:NOTLAW, rules are not important; what matters is whether we are improving the encyclopedia or not. Please demonstrate a problem for which this is the solution. Colonel Warden (talk) 09:08, 3 January 2009 (UTC) I think the principle of the current wording in WP:NOR is OK, but in real cases there are grey areas. For example if one simply says that Shakespeare's Hamlet contains a soliloquy beginning "To be or not to be ..." that's fine. However if one comments at all about the soliloquy, even by a calling it a famous or much-discussed soliloquy, one needs secondary sources. OTOH DGG's "To an historian, a newspaper is a primary source; to us, it is a secondary source" raises interesting issues. If the newspaper is aWP:RS we can use the content of it articles for comment on another subject, but as soon as the newspaper, or a group of newpapers, or newspapers in general become the subject, we need external sources. --Philcha (talk) 09:55, 3 January 2009 (UTC) Well now I'm confused as a newspaper is just a collection of articles. Surely a reporter in Afghanistan, describing some battle, produces a primary source in his account for the newspaper. Whereas a journalist reporting on some drug discovery is a secondary source because he's getting his facts second-hand from a press release or recently published primary research paper. And a op-ed is primary or secondary depending on what you are drawing from it. And so on... Colin°Talk 10:32, 3 January 2009 (UTC) Re: "different subjects use [PSTS] differently", I really don't see why we can't just pick one definition that suits our purposes. I've cited several web sites further up this talk page that contain definitions of PSTS that seem reasonable to me and, more importantly, useful to WP. Either we should come up with a fixed definition or else give up on categorising sources into those groups, and just describe a continuous spectrum from witness/original-writing outwards. In this spectrum model, the closer one is to the primary-end, the less interpretation/analysis/synthesis is to be found in the source, and the more likely it is that any useful interpretation/etc will require expert skills (which WP editors should not attempt to use). Colin°Talk 10:32, 3 January 2009 (UTC) Re: "Does this restriction conflict with WP:NPOV" I don't understand this. Why should one form of source inherently have a different POV to another? How is a restriction on the use of primary material likely to lead to bias? It is much more likely that an editor will run into NOR problems when using primary sources than secondary. Colin°Talk 10:32, 3 January 2009 (UTC) I agree with Colin, the restriction on the use of primary sources is there to discourage original research and does not usually breach an article's neutral point of view. In articles which have a high controversial political content one can see the selective use of primary sources to advance a position in secondary sources. For example From the same RAF Group briefing paper on the upcoming bombing of Dresden: Michael Zezima From Dresden to Baghdad 58 Years of "Shock and Awe "Dresden, the seventh largest city in Germany and not much smaller than Manchester, is also [by] far the largest unbombed built-up area the enemy has got. In the midst of winter, with refugees pouring westwards and troops to be rested, roofs are at a premium, not only to give shelter... but to house the administrative services displaced from other areas... The intentions of the attack are to hit the enemy where he will feel it most...and to show the Russians when they arrive what Bomber Command can do." Norman Longmante in The Bombers page 333: Dresden, the seventh largest city in Germany ... is also far the largest unbombed built-up area the enemy has got ... At one time well known for its china, Dresden has developed into a industrial city of first-class importance and ... is of major value for controlling the defence of that part of the front now threatened by marshal Koniev's breakthrough. ... The intentions of the attack are to hit the enemy where he will feel it most ... to prevent the use of the city in the way of a further advance, and incidentally to show the Russians when they arrive what Bomber Command can do. It is exactly this sort of difference in emphasis that this restriction places on Wikipedia editors when using primary sources, and I think that complements and does not contradict the neutral point of view policy. If someone has an example where the restriction has affected an article's neutral point of view, I think it would help us to reach a consensus on if this policy needs adjustment. --PBS (talk) 11:10, 3 January 2009 (UTC) Agree that examples of alleged "conflict with WP:NPOV" are needed to establish if there's really an issue. DGG seems to be making a selective quotation from the policy to portray an extremely rigid distinction which in the policy is softened by "Primary sources that have been published by a reliable source (for example, by a university press or mainstream newspaper) may be used in Wikipedia, but only with care, because it is easy to misuse them. Any interpretation of primary source material requires a reliable secondary source for that interpretation." If specialist knowledge is needed for any reasonable, educated person to understand the interpretation, then we need a published source plainly making that interpretation. The need for care to interpret sources correctly applies in all cases, and editors can discuss sources to ensure that they're portrayed accurately, but there's a particular need to use primary sources for obvious facts, while finding secondary sources for interpretations to avoid synthesising "wow, that means that...." even with the best of intentions. It's a very useful discipline, not a rigid rule. . . dave souza, talk 11:31, 3 January 2009 (UTC) The separation into primary and secondary sources is artificial. There is not a hard and fast line between them. The ruel we have about "reasonably educated person" is fraught with problems. Consider a case from theoretical physics there are many by the nature of the field. Take General Relativity or String Theory. The best sources are papers published in journals. But those are primary sources! What weend up with is a situation where if that ruel iis followed a paper by Einstein could not be used. But a text book could? The reasonably educated person thing also comes into play. Who is educated enough to interpret such sources? Wouldn't one need to be a theoretical physicist? At least a grad student? Or should we pretend that such education is not needed? I think that no matter how we do this those questions neednanswers.--Hfarmer (talk) 12:55, 3 January 2009 (UTC) Good points. Many sources which are, in fact, derived from true primary sources, are considered "primary" by WP. I recall a long discussion about the use of maps, almost all of which are actually secondary sources, but which are called "primary" by WP guidelines. I also recall discussions on what "specialist knowledge" actually entails. In my honest opinion, the entire definition of types of sources is errant, and a definition of "specialist knowledge" is sorely needed. Perhaps we also need a tag for claims which may not logically follow what is said in the entire source (right now we encourage using short quotes as proof of what a source says, even if the rest of the source belies the single line). Also the entire concept of OR probably needs rework, as it encourages use of really poorly written "reliable sources" for claims which would be belied by examination of what we now call "primary sources." Collect (talk) 13:10, 3 January 2009 (UTC) If a poor secondary source is belied by primary sources, the answer is to find a better quality secondary source, not to introduce our own interpretation based on non-obvious "expert opinion" about the primary sources. . . dave souza, talk 15:08, 3 January 2009 (UTC) That's fair. Though my bigger concern, truth be told, is obvious expert opinion. :) Phil Sandifer (talk) 17:03, 3 January 2009 (UTC) Dave that's not what I'm talking about. What I am talking about in the case of a science is journal articles. They are usually the first reports of a new discovery and thus are primary sources by the strict WP definition. This is the most true in the case of theoretical science such as theoretical physics. Just what would the primary source for the principle of General Relativity be if not a paper on the subject? Einstein's Brain, the brain of another scientist applying it to a problem. In a real sense for such sciences there are no secondary sources. Just primary and tertiary. The reasonable education clause also does not make sense. What is reasonable education to write an article about General Relativity would have to be at least a MS in Physics! Likewise for other specialist topics. But as I recall back in the WP:Expert rebellion the idea of certifying some wikipedians as experts was discounted. It's not egalitarian enough. It may just be that wikipedia has gone as far as it can while not recognizing that specialist knowledge of it's editors needs to be given some part in the process. --Hfarmer (talk) 17:09, 3 January 2009 (UTC) The fact of the matter is that any explicit privilege for experts is a DOA proposal. On the other hand, there's a very, very big difference between requiring credentialed expertise, and actively setting up people who don't understand a topic up as the deciders on the article. Phil Sandifer (talk) 17:16, 3 January 2009 (UTC) Agree with DGG's idea that there is no issue as the fundamental act of summarizing any source can possibly introduce OR if not done appropriately, not just limited to primary sources. A secondary source may be a scathing review of a work (something we can't do from the primary source ourselves), but to take that scathing review and then summarize down as "X considered this the one of the worst films in history" is introduction of OR if nothing about "worst film" is ever stated in the source, despite the implication that is non-obvious to make. --MASEM 13:05, 3 January 2009 (UTC) If a notable critic descibes a film as "one of the worst films in history", that's a worthwhile opinion about the quality of the film. We can include obvious facts about the film, such as a non-judgmental plot summary, but must find good quality critics for opinions about the film. . dave souza, talk 15:08, 3 January 2009 (UTC) That's fine, but my example is that if the critic specifically does not say "worst film" despite it being obvious from the critic's past history that he absolutely loathed the film and ranted on it more than any other, then we, by OR, cannot state that the critic thinks its one of the worst films. That is, we can abuse OR with secondary sources just as much as we can with primary sources. --MASEM 17:19, 3 January 2009 (UTC) I agree with much of the above: the distinction between primary, secondary, and tertiary sources is unclear and serves mainly to confuse people. And it really doesn't make a difference to the core message of the policy: no matter what the *arity of the sources, we can only describe what they already contain; we cannot analyze or interpret any source of any *arity. I suggest that the whole section on *ary sources should be kicked out of this policy into an essay. The caution that primary sources are "easy" to use for OR doesn't need the force of policy, and much of the strife comes from people thinking this caution is an additional prohibition because it is in the policy. Regarding specialist knowledge, I don't know what definition we might reach. If you're discussing the Higgs mechanism, is it specialist knowledge to know what a "boson" or a "gauge symmetry" or an "SU(2) doublet" is? Is the answer the same when you're discussing general physics or mathematical matrices? Anomie⚔ 14:50, 3 January 2009 (UTC) I've posted an example below of of how this poses an immediate NPOV problem. More broadly, we run into major problems with academics and specialists who are notable but not the subject of huge volumes of secondary literature. Given that, in many fields, lit reviews and summaries of scholarship are not common, I am hard pressed to say that, for instance, we should have an article on Tim Brennan, but in that article should not summarize his work and thought. But that is going to require basic summary of primary sources. I recognize the importance of the prohibition against allowing specialists to just claim credentialism as the basis for their novel readings. But surely we can distinguish between a case where specialist knowledge is being used as a justification for one view over another, and a case where the two positions are "Source X says Y" and "I don't understand Source X at all." That is, if nobody, upon reading a source, thinks that a claim is an inaccurate summary (as opposed to simply not knowing that it is accurate), there is no problem. Phil Sandifer (talk) 14:58, 3 January 2009 (UTC) I think those who wish to remove this clause have some fundamental misconceptions about what the policy is saying. Nothing in this policy says that we can not summarize a primary source. In fact, the whole point of this section of the policy is to specifically say we can (albeit with care). The policy allows us to report on what a primary source says (including describing any interpretation, anaysis that the source makes or conclusion that the source may reach). What the policy is telling us we can not do is interject our own interpretation, analysis, or conclusions about the source or its topic into a Wikipedia article. Since we can fully describe what both the original (primary) source says, what any critics say, there is no NPOV conflict. All published viewpoints can be represented in our articles. Blueboar (talk) 16:03, 3 January 2009 (UTC) The problem remains that such descriptions do not seem to me to be cosnsitently verifiable without specialist knowledge. Again, though, my current thinking on this is that we might be best off explicitly distinguishing between situations where there is actually a controversy and where there are merely editors who don't understand the source. Phil Sandifer (talk) 16:55, 3 January 2009 (UTC) There is no NPOV conflict at issue with this clause. I disagree that the confusion comes from misunderstandings about what is or isn't a primary source. The clause reads, "a primary source may be used only to make descriptive claims, the accuracy of which is verifiable by any reasonable, educated person without specialist knowledge." The clause requires editors stick to describing the content in the source, and not slip into evaluating the content in the source. Any analytic, synthetic, interpretive, explanatory, or evaluative claims about the contents of primary sources must be attributed to a secondary source. I submit that "the accuracy of the representation of the analysis, synthesis, interpretation, explanation, or evaluation cited to a secondary source must be verifiable by any reasonable, educated person without specialist knowledge" as well. Perhaps this isn't clear in the present wording. Professor marginalia (talk) 18:01, 3 January 2009 (UTC) Would it help if we were to change the wording to: "a primary source may be used only to make descriptive statements about what the primary source itself says or depicts", the accuracy of which is verifiable by any reasonable, educated person without specialist knowledge." ? Blueboar (talk) 18:19, 3 January 2009 (UTC) Maybe not because it may invite more legalistic hairsplitting about when a primary can also be a secondary, etc. I admit I'm confused at times about what the dispute is here. Is it about what a primary source is? Is it about whether there is a NPOV conflict when primary and secondary sources have different rules? Or is it whether or not the "verifiable by any reasonable, educated person without specialist knowledge" language unduly limits the citing to technical sources? I foresee that this RFC will resolve nothing if we can't even come to a shared understanding of what the dispute's about. Professor marginalia (talk) 18:44, 3 January 2009 (UTC) My issue is that it unduly limits the citing to technical sources, especially because peer-reviewed academic sources - the sources that WP:RS pushes us most towards - are the most likely to be technical. Phil Sandifer (talk) 20:16, 3 January 2009 (UTC) That's why I'd like to remove the whole *ary source thing completely: eliminate the hair splitting by eliminating the hair. As far as I can tell, there are actually multiple issues here. The main question is "What exactly is 'specialist knowledge' and does prohibiting its use in understanding the source unnecessarily restrict the usability of sources?". But since it relies upon the primary/secondary distinction, the perennial background issue of "What exactly is the dividing line?" is involved in the determination of what the main question applies to. The third issue, subject to much long-winded discussion above, has something to do about whether a biographical article citing a reliably published journal article can also cite a self-published response by the article's subject; it's all WP:TLDR for me and I can't really see the connection. Anomie⚔ 21:16, 3 January 2009 (UTC) I do not see any conflict with NPOV. I do think there is a major confusion here caused by language. Specifically, I never liked the metaphor or word "close/closer/closest" in relation to this policy. The play "Hamlet" will in almost any context I can conceive of be considered a primary source, but who are we to say how close or far it was Shakespeare's imagination let alone the actual history of Denmark? Moreover, I think the word "restriction/restrictive" also muddies the waters. Finally, I think that there is a major confusion because many people seem to think of "primary source" and "secondary source" as a thing when it is really an idea. A courtroom transcript, a novel, a newspaper article, an encyclopedia article, these are all "things" but "primary/secondary/tertiary" are ideas about things, not things themselves. Their meaning is not intrinsic but dependent on usage. Now, I think that understanding these ideas in terms of "closeness" has an obvious sense to it but will regularly get us into trouble. I think the only meaningful way to distinguish between primary, secondary, and tertiary sources is to think of them in terms of how they are used, and not "what" they "are". Tertiary sources are used for relatively concise general overviews of some object of knowledge. Secondary sources are used to learn a particular person, or group of people's, views - usually in the form of some kind of argument, involving interpretation or explanation of some object of knowledge, an argument that usually involves analysis, synthesis or both. Primary sources are used as the raw material for some secondary source. NOR prohibits any wikipedian from promoting their own arguments, period. But it makes perfect sense for me to emphasize this when talking about primary sources because primary sources so oftn invite interpretation or explanation. They are usually made available to us precisely because so many others hae forwarded explanations or interpretations of them, or to encourage scholars to come up with new interpretations or explanations. So I see value in emphasizing to our readers/editors that Wikipedia is not the place to do this. That said, I see no problem at all with describing primary sources in our articles, or using primary sources to provide a description of something. I do not see this as a restriction but on the contrary a permission, a prescription: description is okay!! I really do not see the problem here. Our policy is not regulating things, it is regulating peaople - us, and what we do. We can provide descriptions of things, and we can provide explanations and interpretations of things. When we do the former, we are treating something as a primary source. When we do the latter, we are treating it as a secondary source. I don't see the problem. Slrubenstein | Talk 19:23, 4 January 2009 (UTC) Does anyone else appreciate the irony that determining what is or isn't a primary source requires specailast knowledege? Dhaluza (talk) 10:10, 5 January 2009 (UTC) I agree with Slrubenstein who clearly has a grip on this policy. I don't think existing policy needs to be changed. --Gavin Collins (talk) 11:38, 13 January 2009 (UTC) Example Reposted from a few sections earlier: Critical Inquiry, one of the top journals in the humanities, publishes frequently a section called "Critical Response," in which a writer pens a response to a previous essay in the journal, and the writer of the original essay responds in turn. I do not know how many such sections have appeared in the 30+ years of the journal - they were included in almost every issue in the early days of the journal, and still appear from time to time. Some of these are landmark debates about which numerous secondary sources exist - Derrida engaged in a particularly famous one over the legacy of Paul de Man, another raged so widely as to be made into a stand-alone book called Against Theory, and a third, among Wayne Booth, J. Hillis Miller, and M.H. Abrams, is one of the most cited exchanges on deconstruction. Others are less widely cited. To pick a lone example from the latter category, the Autumn 1995 issue of the journal has a response by Richard Shusterman to an article from the Summer 1994 issue by Tim Brennan, followed by a response from Brennan. Although we do not have an article on Brennan, he is cited widely enough that he meets WP:N. So let us imagine the article Tim Brennan. Shusterman's response to him, being published in a prestigious journal, is surely a notable perspective, and thus would be included in Tim Brennan. But Brennan's rejoinder to Shusterman is a primary source for Tim Brennan. And as Brennan's work is specialized, there are obvious problems in using his response - simply put, we can't summarize it as thoroughly as we can Shusterman's equally specialized and technical critique of Brennan's work. (Add to this the fact that Brennan's original essay from 1994 actually quotes Shusterman extensively, and so this problem actually exists for both Tim Brennan and Richard Shusterman). But while Brennan's article is cited 19 times or so that Google Scholar can find, one is hard pressed to find a lot of secondary sources on this particular exchange, as, unlike the Derrida/Searle exchange, it is not so significant as to potentially deserve its own article. This is a NPOV problem. One side of a debate gets disproportionate representation, and a far easier standard of evidence for inclusion. The other side is given a short shrift amounting to "Tim Brennan responded to these critiques." NPOV? No. Not even a little. There are dozens of near identical cases in this journal alone, which is one journal in one academic field. Phil Sandifer (talk) 14:58, 3 January 2009 (UTC) I don't understand your example. Is the original article about the person Tim Brennan (ie an autobiographical piece) or about something else (eg how to change the spark plugs on an 1958 348 V8 Chevy engine)? --PBS (talk) 15:14, 3 January 2009 (UTC) It is about something else, however being in the field of the humanities, it is not about something more or less objective like spark plug changing. In this case, it is about rap music and culture. Phil Sandifer (talk) 15:24, 3 January 2009 (UTC) Surly the article that the Brennan paper would appear in is Rap music rather than Tim Brennan, so then unless the original Brennan paper could be considered a seminal paper that sparked a paradigm shift in how rap music and culture are viewed, I do not see how it can be considered a primary source. --PBS (talk) 15:55, 3 January 2009 (UTC) The norm seems to me that we would normally also summarize a scholar's work and thought in their biographical article. Phil Sandifer (talk) 16:19, 3 January 2009 (UTC) I would agree that the norm seems to be that we would normally summarize a scholar's work and thought in their biographical article. What we don't have to do is go into great detail, or great detail as to his critic's views. I could even see devoting an entire section of a bio article to the subject's views on a given topic, if those views are what makes him notable. In the context of a bio article, reporting on the views of the bio subject is appropriate. However, in the context of a bio article, I don't think we need to go into detail discussing the views of the subject's critics. Such in depth analysis of various views is better placed in an article on the topic about which the subject and critics have views. In fact, as long as the section has a pointer to the "topic article" (where all views should be discussed) I would say there is no real need for the bio article to go into the critics views at all. Some may say that this creates a NPOV issue in the bio article, in that you have discussed the subject's views and not those of his critics (note that, in terms of which NPOV, this is the exact opposite of what Phil is talking about)... but within the context of a bio article, it is appropriate to give the subject's views more weight (and even sole weight) Blueboar (talk) 16:45, 3 January 2009 (UTC) Though the problems in summarizing Brennan's arguments hardly disappear in this case. Phil Sandifer (talk) 16:53, 3 January 2009 (UTC) Perhaps, but we are not talking about how easy or difficult it is to summarize Brennan's views, we are talking about whether doing so is a NPOV violation. My point is that it isn't. Context is everything here. Blueboar (talk) 16:59, 3 January 2009 (UTC) Well, I mean, I would still say that high-profile, peer-reviewed criticism of Brennan's views is a significant viewpoint in the bio article. Especially because, in this case, we're looking at rap music, an area with a ton of academic scholarship. We can't summarize every argument there - they're going to have to get moved to ancillary articles, and the bio articles for the scholars in question are an attractive destination in that regard. But as you point out, in the context of a bio article, the weight is on the subject's views - I would suggest that heavily shortchanging them and avoiding summary is still a NPOV problem. Egregious under-coverage is also a failure to give due weight to all significant viewpoints. Phil Sandifer (talk) 17:02, 3 January 2009 (UTC) Well, I feel this does limit the use of primary sources quite a bit if they can only be used to make descriptive claims. Right now the policy is ambigously worded. I can barely understand it after reading it over several times. Perhaps it should be reworded.Teeninvestor (talk) 17:51, 3 January 2009 (UTC) <ri> Philosophers must have quite a communication problem if the accuracy of a summary of their writings can't verified by any reasonable, educated person without specialist knowledge. Agree that we should be applying the same standards to the incomprehensible writings of both philosophers, but surely any such esoteric language can be discussed and explained on the article talk page, with reference to suitable translations of any jargon. . dave souza, talk 17:54, 3 January 2009 (UTC) Someone mentioned a discussion about maps. I recall reading that thread. Some argued that maps were primary sources and that the simple act of reading the map and using it's scale to compute the distance between two points is OR. Some argued that say having three points on a map with a scale and using trigonometry to determine the distance between points on the map is OR. IMO something like that is just applying what to me is basic math, Trigonometry and Euclidian Geometry. To some people even that is pratically mystifying level of mathematical computation. This leads me back to my example of General Relativity. To some editors if I write that the Potential energy in General relativity is a function of r then cite Einstein that could be called OR. To take this to a totally different topic. I have been involved in the past in a dispute over weather or not different words which have the same meaning as given by a RS can be equated or is that OR? Some say it is some say it isn't. My point and I think DGG's point is that the way the text of the OR policy has been interpreted is overly broad. It has been interpreted by many in a way not in accord with WP:NOTOR. Therefore I propose this. We promote WP:NOTOR from a guideline to a policy with the same weight and force as WP:OR. That is the simplest solution.--Hfarmer (talk) 18:05, 3 January 2009 (UTC) It's an essay, not a guideline, and that would be a pretty drastic step. A quick glance at it shows up a few things that I think could be problematic if it became policy. dougweller (talk) 18:38, 3 January 2009 (UTC) No change needed. I'm frankly not seeing the problem here, folks. So (to use the example) Tim Brennan published something about rap and culture. And someone else writes -- in a highly respected publication with a proper editor in charge of it -- an article that says Tim Brennan's recent paper did a good job on points A, B, C, but strangely overlooked the author's pet idea, point D: that rap music causes people to wear their baseball hats backwards. Nothing in our current rules prohibit anyone from saying "Brennan's paper was criticized as overlooking the effect on the orientation of baseball hats among rap listeners.[ref]" Has someone perhaps confused 'primary source' with 'self-published'? Put me in the "ain't broke/don't fix" camp. WhatamIdoing (talk) 19:15, 3 January 2009 (UTC) I'm not sure you understand the issue - the problem is that Brennan's original paper is a primary source for Tim Brennan, but because it is a specialized paper, summary of it is not allowed in that article (even as the same summary would be fine in an article on rap music or elsewhere). Phil Sandifer (talk) 20:14, 3 January 2009 (UTC) I am not at all sure that the original paper is a primary source for the Tim Brennan article. This is what I was talking about several long threads ago when I discussed the distinction between first/third party vs. primary/secondary sources. I would say it is a first party, secondary source. Blueboar (talk) 23:33, 3 January 2009 (UTC) Nor can we be in any way assured that "summary of it is not allowed in that article." To recap, Shusterman's article here at WP merely alludes to the fact there is a Brennan cite challenging Shusterman, but doesn't say anything about it that requires specialized knowledge as is presented here. Brennan himself doesn't even have an article at all. Another editor has already weighed in on this example, saying that the debate and rebuttal between Shusterman and Brennan aren't specialized after all-regardless of which is a primary or secondary source. This is a nonproblem, and not a good justification for changing the policy as it is now. Professor marginalia (talk) 00:56, 4 January 2009 (UTC) I disagree, simply because in the current language overwhelming power is given to veto - all it takes is one non-specialist to justify large eviscerations of content. Which is the underlying problem here - the non-specialist clause vests vast power in those who do not understand the topic they are editing. This is undesirable, to say the least. Phil Sandifer (talk) 16:26, 4 January 2009 (UTC) "All it takes is one non-specialist"? Does it take just one objector to say a claim violates NPOV to remove it? Or one claimant that a published source is unreliable? With these "vast powers" given in the policy for over three years, perhaps we could find some examples of it being implemented in this fashion? PS, were disputes in Derrida article the spark triggering your concerns over this clause? Professor marginalia (talk) 17:38, 4 January 2009 (UTC) Yes. That is the consequence of "any reasonable, educated person." The threshold of "any" means that the existence of one who cannot verify is sufficient to remove the line as OR. Which sets it apart from NPOV or RS, which lack the demand for unanimous consent. Phil Sandifer (talk) 17:48, 4 January 2009 (UTC) So "any reasonable, educated person" is a demand for "unanimity of editors"? I haven't seen the policy implemented this way. Have you? Professor marginalia (talk) 18:23, 4 January 2009 (UTC) Groaning Blueboar, I didn't address the issue of whether it's a primary source because I don't know enough about the specifics for the (hypothetical?) paper. But you are absolutely right that it could be considered a first-party secondary source (e.g., the class that all journalist-on-the-scene news articles fall into). Prof, I can't image why anyone thinks that a summary is not allowed. What exactly is a "descriptive claim" of a paper if it's not a summary of the paper's contents? Solving Would WP:PSTS benefit from either (1) a statement that editors should not confuse these terms with wikt:first party and wikt:third party and/or (2) a table showing how these intersect? I have something like this in mind: First party Third party Primary source Notes taken by a journalist during an interview with a person accused of a crime A court order about the alleged crime Secondary source News story based on those interview notes News story about the court order Tertiary source Another person writes about the news story Would something like this (with more and/or better examples) be helpful? WhatamIdoing (talk) 02:51, 4 January 2009 (UTC) (who can't believe how many times this question has come up in the last few months) The above definition of primary and secondary disagrees with any that I can find elsewhere. Both the interview notes and the publication of the news story (based on those notes) are primary sources. The article by a different person discussing the news story is a secondary source. The journalist writing later (in his memoirs, say) about that news story would also be a secondary source on the news story, but a primary source on his opinions. The "journalist-on-the-scene news articles" are all primary sources in any definition I can find. I'm happy to be corrected but please can folk cite some authoritative source (preferably online, but you can quote it if not) that back up the definitions used on this talk page. I feel that the above takes a "X based on Y so X is +1-ary more than Y" approach which is not only too simplistic but leads to one running out of -ary categories quickly. For example, if the journalist had recorded the interview to tape and then made notes from the recording and then wrote his article from his notes, would the news article be tertiary? Perhaps some people do define PSTS with such a rule, but I can't see how it is of any use to us. The number of stages a piece of original work goes through prior to its publication must be irrelevant to us. There's too much original research going on in folk's examples and definitions IMO! Colin°Talk 10:32, 4 January 2009 (UTC) I agree that such a simple chart is would not be the right approach for a policy. I introduced the idea that there was a distinction between first/third party and primary/secondary source purely for talk page consideration... I don't think it needs to go into the policy. My point was to question the assumption that anything written by the subject of a bio article should automatically be considered a primary source, and offer a new way to look at this issue. Blueboar (talk) 15:53, 4 January 2009 (UTC) All sources are primary for something: even an school textbook (normally classed as a tertiary source) is a primary source for the fact that John Smith wrote a textbook on Biology in 1996, which had chapters on A, B and C. In an article on John Smith, it would be a primary source for this fact. In an article on biology it would be a tertiary source on aspects of biology. A review saying the biology book was very good but missed out D and E would be a secondary source on aspects of the book and its missing contents (for use in John Smith's article) but probably wouldn't be any use in the biology article. This sort of thing is why saying "X is a -ary source" isn't a good idea without context -- what facts are being drawn from it. Colin°Talk 16:10, 4 January 2009 (UTC) Colin, I'd be perfectly happy with much better examples, if such a thing would be helpful (perhaps, however, on a No original research examples page). I think that the confusion between "primary" and "first person" should be addressed. WhatamIdoing (talk) 00:40, 7 January 2009 (UTC) It might not be all that important to explicitly pin down this idea of "first party" vs. "third party" vis-a-vis primary and secondary. After I did a bit of research on the issue, it turns out that "third party" is often presumed in the idea of a secondary source. Old Dominion University library, for instance, defines it as follows: "Secondary Source: second-hand report or review of original research that is written by someone other than the original researcher." Here are a bunch of definitions: [15]. To me they tend to lead towards the general expectation that a secondary source is written by someone other than the author of the primary source. ... Kenosis (talk) 01:00, 7 January 2009 (UTC) Generally speaking, I support the position that secondary sources should determine the due weight that primary sources (writings by article subject, insider accounts, etc.) should be given. So, whether or not to quote a primary source, and what parts of it to quote, is a decision that should not be taken by Wikipedia editors. We have many, many WP:UNDUE problems in articles where WP editors have made such decisions. In the Brennan example, if no third-party source quotes and evaluates Brennan's rejoinder, then neither should we. For that matter, if the whole Shusterman–Brennan dispute remained uncovered by sources secondary to it, then perhaps we shouldn't quote Shusterman either in Brennan's article, but just mention that there was a dispute between them in such-and-such a publication (descriptive, and verifiable without special knowledge), give an external link to the articles and rejoinders if they are online, or include them in "Further reading", and/or include their respective opinions in our article on the topic which they were debating, and on which they are expert commentators (i.e. authors of secondary sources). I don't think the NPOV concern is all that valid, or that changing primary-source policy is the way to address it. Notable people are to some extent at the mercy of commentators, i.e. the mirror the world holds up to them. If all published commentators are critical, then that's tough! If the only one defending the notable person is the notable person themselves, their self-serving defence should remain off-limits to us as editors, unless it's been quoted by other commentators. But let's be clear that in real life, if the person attacked has a shred of credibility, some commentators will defend them, or at least paint a different picture of them, and we can quote such commentators. All part of writing a "Reception" section. Jayen466 20:15, 4 January 2009 (UTC) This is perhaps workable for figures in the upper echelons of notability. But in those lower echelons - when we have enough sources to establish notability and perhaps even a few more - we run into problems with the number of commentators on a given side. It does seem to me that a subject's own defense of themselves is always a significant viewpoint, however. Even when that view lacks credibility, it still seems to me to have significance. Phil Sandifer (talk) 20:27, 4 January 2009 (UTC) Pardon my persistence, but do you have an example to show where the current policy is "unworkable" for describing a subject's self-defense against notable criticism? Thanks. Professor marginalia (talk) 20:48, 4 January 2009 (UTC) I am having problems following Phil Sandifer's argument. Above he wrote, "The norm seems to me that we would normally also summarize a scholar's work and thought in their biographical article." Surely this is not the norm. Even if a person passes a certain threshold of notability (I believe Brennan does), surely we should not go by the norm. I suspect that the length and level of detail of biographical articles on scholars is bimodal, and we should have two "norms" - or, it follows a gradient. either way we should not have one norm. My sense is that the more notable a scholar, the more likely there are a range of secondary sources - biographies, intellectual histories, and so on - about that scholar. In such cases, we draw on those sources to provide a fitting account of the scholar's work, that is in proportion to her notability. Other scholars (say, Brennan) are less notable, and as Phil points out there are fewer sources. In such cases, the article will be shorter and less detailed. Since we can use primary sources for uncontroversial descriptive claims, I see no problem with using the dust jacket of Brennan's books, or abstracts of his articles, or his web-page, to say "Brennan has researched and written on x, y z." Phil Sandifer points out that there are no good secondary sources on Brennan's work, or his views in certain debates or exchanges. Okay, Phil, do you want to write one? Well, great! Write an article and submit it to Critical Inquiry or Representations or Daedalus or whatever and see if they will publish it. But do you want to write such an article here? No, Phil, that would violate NOR. Slrubenstein | Talk 21:08, 4 January 2009 (UTC) Agreed. In addition, the alleged contradiction between NPOV and NOR here seems spurious to me; primary sources are generally open to a much wider degree of interpretation (and thus original research/author bias) than secondary sources. As such, the restriction on primary sources enhances NPOV rather than conflicts with it. Jayjg (talk) 22:58, 4 January 2009 (UTC) My issue here is that it seems to me an egregious creep of scope of NOR if it has grown now to encompass uncontroversial summaries of sources. I remember quite well the original scope of NOR on these matters - it was intended to prevent "novel interpretations" of primary sources in history. The shift to it now, apparently, blocking summary of the basic points of academic articles - even in cases where nobody actually disagrees about what the article says. That is a dramatic shift - one of the largest expansions in scope of a policy I can think of in my time on Wikipedia. If such a massive shift has occurred, I would hope it has been discussed and a clear consensus for that shift has been demonstrated. But in the absence of any actual evidence for such a consensus, I am forced to believe that this has been a sneakily inserted undiscussed scope creep. Phil Sandifer (talk) 23:15, 4 January 2009 (UTC) HUH? where the heck does the policy even come close to "blocking summary of the basic points of academic articles - even in cases where nobody actually disagrees about what the article says"? Blueboar (talk) 23:56, 4 January 2009 (UTC) I don't see any problem with using this example of an exchange between Richard Shusterman and Tim Brennan to create an article on this specific controversy, if it's been mentioned in other sources (and so notable). Then within that article, the responses from all parties would be equivalent. The issue seems to be, that in a biography, anything the subject writes is being considered a primary source. I'm not sure that's accurate. A primary source presents something new, never before seen, a new fact. Secondary sources comment on those primary facts. If I, comment on my own book to explain, clarify, analyze, defend, it seems like I am creating a secondary source, even though I am speaking in the first person, about my own work. Book reviews, are always secondary. Right? Wrong? Apple? Wjhonson (talk) 09:10, 5 January 2009 (UTC) I share Blueboar's reaction. Phil Sandifer now says that his problem is that the policy prohibits summarizing primary sources. I just do not see anywhere in the policy where it says this. The policy explicitly states that we can use primary sources to make descriptive claims, isn't a summary of a primary source a description of it? It seems like this debate is over. Problem solved. Slrubenstein | Talk 15:42, 5 January 2009 (UTC) Just to comment on the original request: Primary sources should be avoided for anything but verification of objective facts and should be quite explicit in the text which primary source was used. If there isn't a secondary source interpretation of the "summary" desired, that may be an indication that the concept is not sufficiently notable to have a neutral and well written article on the topic. This, in my mind, is part of why we have WP:N-it rules out original authorship since it demands that editors use established thought instead of interpreting. If there aren't reliable secondary sources available, why do we have an article about it? WP:IAR can be invoked, but if we are relying on original summaries of primary sources, my knee-jerk reaction is to describe it as original synthesis and inappropriate for wikipedia. SDY (talk) 19:17, 5 January 2009 (UTC) IMO, that pretty much says it in a nutshell w.r.t. whole articles. In addition, the concept also works well for specific points WP editors may wish to make for the reader within an article. If there are no secondary sources, use the rule for primary sources and describe what the source says in a way that a hypothetical "reasonable, educated person without specialist knowledge" can look at the source you've cited and conclude that what you've said reasonably reflects what's in that source. If it requires specialist knowledge and there are no secondary sources that sum up what's in the source, it should raise a red flag to double-check the source's standing per WP:FRINGE and WP:RS. If it passes muster w.r.t. WP:FRINGE and WP:RS, one should proceed to use the source without apology and attempt to work out, on the relevant article talk page, any disagreements about the use of the source or about how it's represented in the article. Lacking the ability to arrive at a consensus, there are, as most everybody here already knows, additional options such as bringing it up at WP:NORN, WP:V/N. ... Kenosis (talk) 22:32, 5 January 2009 (UTC) Which is exactly what I have been saying for the last three weeks. Blueboar (talk) 22:42, 5 January 2009 (UTC) Example II:A controversial case It is in controversial articles that WP Policy get's tested tot he limit. Let us consider an old controversy which isn't controversial anymore. It features contentiousness and expert editors mixing with people who argueably could have no idea what the article is really about. Loop Quantum Gravity A bit of proof that this policy has had considerable creep over the years. A long time ago on wikipedia I was involved in a controversial article called Loop quantum gravity. This had it all I'm telling you. It had deeply entrenched editors such as myself and User:Lumidek and many others editing and editing like mad. There were accusations that LQG was pseudoscience, and fringe, and crackpot all over the place. It even got a bit personal. long sections or articles were created and delted and created which had peoples "Objections to the theory of Loop Quantum Gravity" In those days as this archive of the conversation demonstrates the policy OR and SYNTH etc. NEVER came up at all! Consider how much of what was said there and is still in that article would have been called OR and SYNTH and removed based on the way peopel apply those policies now. Even if specialist argued about why such and such was relevant. It would not matter. Non specialist will always outnumber specialist. A toatl neophyte could show up at that articles page and delete vast ammounts of good information because it looks like WP OR and SYNTH that must have been hard to do.... While a specialist knows that any theoretical physicist would immediately connect A to B. Basically I see allot of this going on. People who are educated but not specialist go in and becase they are very educated do not yield to someone who's education, or lived experience makes them inherently more knowledgeable about a given topic. They don't see how A is connected to B because seeing that connection requiers training in a given field. It may not even be something that is in an RS... like in physics much of learning it is acquireing a certain intuition, this is true in many other fields people specialize in. I hope that people see where I am commin from on this. We have to make some hard and fast provisions for specialist to come here and write articles without having to speen time building up the expertise of strangers. --Hfarmer (talk) 20:48, 6 January 2009 (UTC) Times do change... It might be helpful for everyone to note something that was mentioned in a recent arbcom case... while the case was specifically about another issue entirely, the arbitrators noted some basic principles that do relate to what we have been discussing here. Note the top one onAcademically demanding subjects and the two or three below it that relate to experts (and one specifically on NOR). Blueboar (talk) 05:20, 8 January 2009 (UTC)

On cherry picking and gun jumping

Rather than getting bogged down in definitions, I'd like to focus on the practical problem. In medical articles, it is very common for people to want to insert a passage dealing with some off-the-wall idea that hardly anybody takes seriously, justifying it by saying that it comes from a reputable source. It's also common to want to alter an article to reflect the latest provocative paper. Those of us who work in science know that on many questions, it's possible to find reputable sources to justify nearly anything -- just by statistics, if you have enough studies, a substantial number of them will come to wrong conclusions. How do we defend against this? This is a genuine problem that comes up every day and will rapidly turn Wikipedia's medical articles into crap if we don't have a principled way of dealing with it. In practice, the people who maintain medical articles mostly follow the policy that I wrote out, and I don't see any realistic alternative. It would be nice if WP:NOR would back us up. Looie496 (talk) 18:43, 12 January 2009 (UTC)

Can you give an example so I know what I'm looking at? Phil Sandifer (talk) 18:45, 12 January 2009 (UTC)

See Talk:Schizophrenia#include early intervention section for the issue that brought me here. See also Talk:Caffeine#Pregnancy for a similar issue. Looie496 (talk) 19:10, 12 January 2009 (UTC)

I appreciate that problem with medical writing. I'm not sure that "originality" is the issue, though – in the situation you describe the material is published, but we want to exclude it for other reasons. I usually argue against these sorts of inappropriate additions using the "due weight" section of WP:NPOV. It may be that in the context of a medical article, the due weight is zero for a new study that is not supported by any other studies and is not covered by reviews. But in other fields the due weight of a newly published paper may be higher. — Carl (CBM · talk) 18:47, 12 January 2009 (UTC)

Here is a dispute illustrating a problem involving original research, primary research, peer review, NPOV, and all the rest.[16] I'm going to be delicate describing the dispute because this is a BLP and its subject was editing in his own WP article. We had the situation of the second of two primary sources, both peer reviewed, being offered as a study confirming the findings of the first. As I said, these were primary sources, peer reviewed. The author of the study was lending his own first hand description of both articles on the talk pages-a qualified expert in every sense. In terms of solid credentials of the authors and academic peer review of the papers, they'd rank much more impressively than most secondary sources would do. However, the editor (also author of the first paper) was attempting to impart a conclusion, an opinion, from comparing the two primary sources. The fact that the two papers are primary research and peer reviewed gives them credibility over secondary sources in academia where forming opinions or conclusions from them is welcome. It is not welcome at WP! The opinions or conclusions that can be drawn from primary sources must come from secondary sources. Professor marginalia (talk) 19:40, 12 January 2009 (UTC)

The first was an excellent example, Looie496. The conclusion the editor made was not the conclusion given in the abstracts of studies. Perfect example of how carried away things can easily get with primary sources! The second example illustrates a problem with WP:Undue, but not original research. Maybe I didn't read it carefully enough, but it doesn't look like this was a primary source misuse at WP, though maybe one in the secondary source. Professor marginalia (talk) 19:56, 12 January 2009 (UTC)

The misapplicatin of WP:NOR is a common problem... I have seen people cite the PSTS section for the arguments that are really WP:V/RS issues, WP:NOR issues and even WP:N issues. One thing that the NOR policy does not do well enough is explain exactly what we mean by Original Research ... as used in this policy, the term does not apply to published research that may be original... it applies to our own ideas or research. In other words it does not apply to our sources, it applies to what we write about the sources. We need to get back to that. Blueboar (talk) 20:33, 12 January 2009 (UTC)

One way to deal with problems like this is to introduce citations to sources that contest the validity of the source in question. There may still be undue weight issues even in this case though. Ultimately authors need to come to consensus about what sources they consider reliable. Dcoetzee 22:27, 12 January 2009 (UTC)

When you have dozens of peer-reviewed articles that all express subtly or not-so-subtly different viewpoints, only a expert can integrate them in a sensible way. Having non-expert Wikipedia editors trying to express all the viewpoints with appropriate weight just leads to an incomprehensible hash. There's really no workable option except to rely on review papers whenever they are available. Looie496 (talk) 22:41, 12 January 2009 (UTC)

No option apart from editing articles on subjects that you are familiar with? — Carl (CBM · talk) 22:51, 12 January 2009 (UTC)

Ah, if only one could get rid of those ignorant other editors :-) While WP is the encyclopaedia that "anyone can edit", anyone really does edit. Any idiot can find a PubMed abstract on Google. At least with a review, we have the opinion of someone we know is an expert, imperfect though they may be. The whole purpose of a literature review is to conduct the kind of research, evaluation and synthesis that we are prevented from. Colin°^Talk 23:02, 12 January 2009 (UTC)

There's no need to get rid of anyone. But we do need to have a core mass of people who know something about the topic of each particular article. Apart from those who follow a few "controversial" topics (which are a tiny fraction of all articles), most editors I have met have no difficulty recognizing the benefits of staying close to the existing literature. The degree to which it's admissible to do a literature survey and summarize the results varies widely between fields on WP – not all fields are like medicine in favoring review articles. — Carl (CBM · talk) 23:17, 12 January 2009 (UTC)

Carl, you're way off on this suggestion. No option. And it's often when we have good editors working on subjects they aren't familiar with that the problems are identified. It's impossible to ask wikipedians to adequately or accurately determine the consensus or weight of opinion of topics in medicine and science by wading through published primary research. Good gawd, what a mess that would be. Professor marginalia (talk) 23:09, 12 January 2009 (UTC)

I have been under the impression for a long time that the only way to determine the consensus of opinion in various areas of science is to be broadly familiar with the contemporary literature. (Not that this is directly related to NOR). If I am not familiar, in a somewhat reasonable sense, with what has been published in a particular area, there's no reasonable way by which I could decide whether some claim was in agreement with the consensus of the literature or not. Putting blind faith in the accuracy of review articles carries all the pitfalls of blind faith. — Carl (CBM · talk) 23:17, 12 January 2009 (UTC)

Quite a package of issues here!

• Part of the instance raised by Looie496 (18:43, 12 January 2009) is about the limits of WP:RS. Why Current Publication Practices May Distort Science regards the eagerness of mags and author for spectacular new ideas as a reason for the high "mortality rate" of peer-reviewed articles - in medicine, the field Looie496 was concerned about. In paleontology artciles I prefer sources in the 5-10 year old range for that reason, and when using more recent sources give the date and use more cautious phrasing, e.g. "In 20nn X concluded that ..."
• Re Looie496's "no workable option except to rely on review papers whenever they are available" (22:41, 12 January 2009), it depends who's writing the review. E.g. you'd get very different opinions in review papers by Caron and Butterfield about the "Halwaxiids". I agree with CBM (23:17, 12 January 2009) about the need for editors to know the subject reasonably well and to avoid "blind faith in the accuracy of review articles".
• [17], quoted by Professor marginalia, looks like a completely different issue. Unfortunately the third of the links in that discussion showed me an empty page. Let's suppose for the sake of argument that the articles said fairly similar things. It's quite likely that a WP editor who understood the subject could validly write that "X, Y, and Z agreed that beneficial mutations had more chance of becoming fixed in sexual than asexual populations (all other things being equal)". OTOH X, Y and Z might have slightly different reasons for coming to this conclusion, and if the reason were relevant to the subject of the WP article, the differences would also have to be mentioned. The only way round this type of situation is discussion between knowledgable editors. It's impossible to apply WP:NOR, WP:SYN, etc. absolutely literally because WP articles have to select, summarise and paraphrase - partly for brevity, partly to make technical subjects intelligble to non-specialist readers, and partly to avoid copyright problems. --Philcha (talk) 00:10, 13 January 2009 (UTC)

Carl: "I have been under the impression for a long time that the only way to determine the consensus of opinion in various areas of science is to be broadly familiar with the contemporary literature." What you're proposing, that wikipedians can wade thru primary research to determine to what degree each particular work is currently generally accepted in the field, is pretty close to original research of kind. Anyway, no. The way we as editors can best determine the level of acceptance scientific or medical ideas have is to consult those secondary and tertiary RS which write about it. Professor marginalia (talk) 01:14, 13 January 2009 (UTC)

Philcha: Re Desai. Not so, not at wikipedia. The policy is, the claim, "Paper B validated Paper A", needs either 1) a source that says, "Paper A is validated by Paper B" or 2) Paper B to say, "this validates Paper A". But if you're saying, "Paper A's conclusion was X" and "Paper B's conclusion was X", then wikipedians can be the first to claim Paper B confirms Paper A? No. Paper B can conclude "Socrates was a man", with Paper A earlier concluding, "Socrates' daughter was a man", but one can't accurately say the analysis in Paper A was confirmed by Paper B. Professor marginalia (talk) 01:14, 13 January 2009 (UTC)

C'mon, Professor marginalia, Socrates' daughter is a straw person of indeterminate gender, and "man" is so ambiguous that it's impossible to use it for any clear and obvious deduction. OTOH the abstracts of the 2 Desai papers (I'd have read the full text if editing an article) said much the same thing except that one explicitly cast doubt on one of the theoretical models and the other only hinted at this. So it would be fair to say that Desai's opinion was mainly the same in the 2 papers - unless the development of Desai's opinion on the theoretical models was relevant to the WP article. That apart, you could translate one of the abstracts into the other with the aid of a good dictionary. --Philcha (talk) 09:36, 13 January 2009 (UTC)

You're at a disadvantage to sort the dispute now - the Snoke dispute was confusing then, and all the links to the related articles don't work . The comparison wasn't between the two Desai papers. It was between those two and a third, earlier, and highly controversial paper that was alleged to imply something about irreducible complexity. Professor marginalia (talk) 15:01, 13 January 2009 (UTC)

(←) "What you're proposing, that wikipedians can wade thru primary research to determine to what degree each particular work is currently generally accepted in the field, is pretty close to original research of kind." As long as they don't try to put their own conclusions into the article, editors should read broadly in the fields they edit in and get a sense of the literature. NOR does not proscribe editors researching the sources, NOR only discusses editors actually writing their own opinions into the wiki articles. Talk page discussion is not covered by the OR policy, and editors are free to use their broader knowledge when making editorial judgments. — Carl (CBM · talk) 02:05, 13 January 2009 (UTC)

I didn't say that Wikipedians can wade thru primary research to determine to what degree each particular work is currently generally accepted in the field. I discussed whether specific authors X & Y agreed with Z. I agree with your point about identifying consensus, in fact it's such a can of invertebrates that I avoid it unless 110% clear. --Philcha (talk) 09:36, 13 January 2009 (UTC)

It sounds nice when you say it that way, but go to autism and try to get a handle of the question of whether mercury plays a role in causing it. Or… well, I could go on listing things, but it's just hard to really grasp the problem if you haven't tried to deal with a topic with a large and contradictory literature, and lots of editors with strong feelings about specific points who say "the article obviously needs to say X, here it is documented in a refereed journal paper!" Looie496 (talk) 02:24, 13 January 2009 (UTC)

Yes, the autism-mercury issue is an interesting and important issue, due to the widespread publicity given to the alleged connection. It's mainly a WP:V#Reliable_sources, WP:RS and WP:WEIGHT analysis, and a very difficult one at that. But one thing that should definitely be ruled out is to have WP editors parsing through the primary-source research to arrive at article content, no matter which side of this issue those sources may seem to support. ... Kenosis (talk) 02:40, 13 January 2009 (UTC)

Apart from researching sources, how would we want people to arrive at the article content? And editors are very correct that if something is included in a refereed journal paper then WP:NOR and WP:V have no objections to including it in the article. There may be editorial issues, or questions of undue weight, but there's no issue with NOR provided the claims are kept descriptive. — Carl (CBM · talk) 04:50, 13 January 2009 (UTC)

Yes. In this particular instance, the editors at autism apparently have chosen to sum it up according to reliable secondary sources. Here is an earlier permutation of the issue. Since then, the users participating in that article have gotten a firmer handle on the issue, which is presently mentioned in the second lead paragraph and at the end of Autism#Causes, relying on secondary-source summaries of the alleged vaccine connection. (See, e.g., current footnote 57.)... Kenosis (talk) 05:32, 13 January 2009 (UTC)

How do they know that these secondary sources are not themselves biased or out of agreement with the consensus of experts in the field? But, on a different point, medicine is somewhat unique in its use review papers. The humanities are not at all the same. — Carl (CBM · talk) 13:32, 13 January 2009 (UTC)

How do they know? They don't. They were applying a reliable-sources analysis and WP:WEIGHT. And (in the end at least) they avoided cherrypicking the primary sources, just as they should have according to the policy, since there were plenty of secondary sources at their disposal, summary-type secondary sources that they then proceeded to use in preference to primary sources in order to sum up the issue for the reader of the article. All in accordance with existing policy, AFAICT. ... Kenosis (talk) 18:18, 13 January 2009 (UTC)

If someone doesn't actually know that what they are adding is right, they really shouldn't be adding it at all. Choosing the reliable secondary sources is no simpler than choosing the reliable primary sources, especially since bias in secondary sources can be more difficult for untrained readers to notice. — Carl (CBM · talk) 19:21, 13 January 2009 (UTC)

I'm confused - the discussion seems to veer further and further away from any original research issue. Potential bias, reliability, or factual accuracy of sources--these potential problems aren't meant to be addressed here in the NOR. Professor marginalia (talk) 19:37, 13 January 2009 (UTC)

But these are exactly the issues that the language about primary/secondary sources addresses. The arguments above against the use of primary sources all refer to the supposed inaccuracies that come from their use. The arguments are not, particularly, about "originality"; the parts of the arguments that are about originality have little to do with the distinction between primary and secondary sources. — Carl (CBM · talk) 20:14, 13 January 2009 (UTC)

Looie:"It sounds nice when you say it that way but..it's just hard to really grasp the problem if you haven't tried to deal with a topic with a large and contradictory literature". Nuff said. It's not just a naivete about editing controversial subjects at WP, it's naivete of the field of science/med itself. That's not what their academics do either. It's not like it's this simple a thing to derive the "consensus" in science by simply conducting a "primary" lit review! Professor marginalia (talk) 05:11, 13 January 2009 (UTC)

I agree with these comments. In controversial topics even review papers need to be used with care, as much may depend who wrote them. That's why I think knowledge of the field and the players is important (see the "large and contradictory literature" at Halwaxiid, and see Orthrozanclus for a bit of a spat in public), and why it ultimately depends on the judgement of editors who know the subject. --Philcha (talk) 09:36, 13 January 2009 (UTC)

I don't know much about "naivete about editing controversial subjects at WP", but I've had plenty of experience with WP editing. And in my field it is perfectly reasonable to do a literature review to find out what people have written on some topic. I still haven't seen any other proposed way (on this talk page) to judge consensus in a field apart from being familiar with the literature in that field. — Carl (CBM · talk) 13:32, 13 January 2009 (UTC)

We are just editors here. We are often asked to judge what weight to give some claim by examining how much weight it receives in the best published secondary literature. But this isn't directly equivalent to the process in which discoveries take hold in the field of science and medicine. The peer reviewed publication of primary research is just one of many levels of review most new research findings will undergo. Professor marginalia (talk) 15:20, 13 January 2009 (UTC)

Attempting to forbid the use of all "primary sources" in medical articles, e.g. RCTs, is ridiculous. In some areas, e.g. anabolic steroids, there are few large scale RCTs, but there are tons of review papers, and they sometimes draw contradictory conclusions on various topics. Because of their scarcity, the good RCTs often get published in prestigious journals, e.g. NEJM, while the reviews have to settle for lesser venues. The common sense thing to do in these cases is to label the source accordingly in the wiki text: "A RCT on nnn patients determined that...", "a year YYYY review that surveyed XX studies concluded..." You can't prescriptively solve all these issues with a paragraph in a policy that's supposed to apply equally well to math, paleontology and medicine. Xasodfuih (talk) 15:40, 13 January 2009 (UTC)

I do wish people would read the policy... at no point does the policy "forbid" the use of all primary sources. It does set limits on what we can say based upon primary sources... ie how we can use them... but this policy does not forbid primary sources. Blueboar (talk) 16:05, 13 January 2009 (UTC)

Blueboar is correct. Note, for example, that the article on hyodeoxycholic acid is written in complete accordance with this policy. ... Kenosis (talk) 18:03, 13 January 2009 (UTC)

For more obscure topics, e.g. hyodeoxycholic acid, there may be no review paper in existence. Usually the best review in this case is the introduction section of a recent paper on the topic. Xasodfuih (talk) 15:56, 13 January 2009 (UTC)

If there is no review paper, I would have to ask whether something like hyodeoxycholic acid is all that notable. But, that is an issue for WP:N and not for WP:NOR.

As far as this policy is concerned, there is no ban on citing the introduction of a recent paper... This policy does not forbid any source. This policy isn't about sources. It is about what we can and can not say about a topic. The fact that there are no reliable secondary sources that discuss hyodeoxycholic acid does not mean we can not discuss hyodeoxycholic acid, it simply means that there is a limit to what we can say about hyodeoxycholic acid. Blueboar (talk) 16:21, 13 January 2009 (UTC)

It seems to me that if parts of the introductory section of a paper give a general description of a subject rather than specifically discussing experimental results than those parts of the introductory section are a secondary source, rather than a primary source. Moreover, as Xasodfuih says, these are often the only sources available for somewhat obscure topics. For example, the first page of this paper [18] is clearly acting as a literature review rather than presenting novel information (except for the final sentence). The remainder of the paper presents experimental results. — Carl (CBM · talk) 16:56, 13 January 2009 (UTC)

The point I am trying to make is that this policy does not (and should not) "forbid" discribing the research paper or the experimental results... it simply limits what we can say about the paper or those experimental results. Neither the paper, nor the results are Original Research as that term is used in Wikipedia... they can't be since they exist in a published source that is external to Wikipedia. What might or might not be Original Research is what we say about the paper or the results. Because the paper is a primary source, there is only so much that we can say using that source. Again... WP:NOR is not about sources... it's about how we use sources... its about what we write in our articles.

Now... there may be other issues involved in mentioning a given source. The source may be considered unreliable, the paper might not be really relevant to the article's topic, mentioning it may give undue weight to a fringe concept, etc. But these are not WP:NOR issues. As far as WP:NOR is concerned, the paper can be used, but we are limited in how we use it. Blueboar (talk) 17:47, 13 January 2009 (UTC)