Wikipedia talk:Content review/workshop/Archive 1

Page contents not supported in other languages.
From Wikipedia, the free encyclopedia

Radical streamlining?

I don't quite see how that description fits this. And I don't really see the point of this to begin with -- not to say that there is no point, just that I don't understand it. You may want to offer some additional rationale at the top of this talk page, especially with regard to how this process compares with "good article" nominations.

Equazcion • argue/improves • 10:08, 10/1/2007
Ah, sorry. Insofar as it might supplant GA it's radical streamlining. Marskell 12:04, 1 October 2007 (UTC)

This idea seems rather stupid, IMHO. The last thing we need is a fourth "review" option out there. I just read something over at WT:FAC last night about the backlog there being much higher than it was, and many of those regulars are seeing a decrease in FAC reviewers as well. So I think the biggest problem that plagues all of the existing review avenues (WP:FAC, WP:PR, and WP:GAN) is a general lack of editors interested in reviewing articles. So adding a fourth review option out there isn't going to help; all that's going to do is to suck more reviewers away from the three areas that we currently have.

Infighting amongst FA & GA isn't going to help this, either. FA isn't going anywhere, and neither is GA. The two camps need to come together and find some common agreement in the middle, and decide how best to fix the greater issue of article review on wikipedia, and decide how PR fits into the picture as well. Dr. Cash 19:08, 1 October 2007 (UTC)

No more review options, I agree. A.Z. 00:52, 2 October 2007 (UTC)
Guys, I said insofar as it might supplant GA. No, of course, simply starting a third structure doesn't make sense. Call it "back to GA's roots", if you like. Marskell 11:15, 2 October 2007 (UTC)

Good articles

How does this idea differ significantly from WP:GA? There is no point having a new process almost identical to an existing one. If you have ideas to improve GA, then suggest them over there, but starting a new process is a bad idea. --Tango 21:54, 1 October 2007 (UTC)

Actually...

I'm changing my vote somewhat. I do see a benefit to something like this. GA seems to be a very strenuous standard to live up to (especially for a standard named "good" rather than "great" or "outstanding"). There are plenty of articles that are generally trusted and accepted as having good information, yet don't meet the GA standard because of some formatting, less-than-great writing, or other pedantic issues that really don't matter all that much when you consider what it takes for an article to just be useful.

Marskell, if you want people to accept this, I would first of all remove the template format from the page. Also post a sample of the tag you'd like used for nominating articles -- and make it something rather lean and nondescript. I'd also try and word the page a bit more simply, and give some more rationale with emphasis on contrasting this with GA and FA. Perhaps something along the lines of this being an intermediate step between "nothing" and GA -- although that's just a suggestion.

Equazcion • argue/improves • 04:39, 10/2/2007

General review

General review

The General review is intended to encourage and identify competent content with a minimum of bureaucracy. It has no subsidiary pages outside of template space. If an editor feels that they have updated a page to the point that a general reader would be reasonably satisfied with the coverage, they may nominate in one of the categories below. The article should not neglect major aspects of a topic, have sound prose, and cite its sources. Nominations need only a blue link and a summary sentence.

Any editor may choose an article from the list: remove it from this page, start an article talk thread, and leave a user talk note with the nominator. Reviewers are encouraged to edit the article directly if they see room for improvement. Once the reviewer and nominator are both confident that the coverage is competent, the successful General review template may be placed at the top of article talk. The template should be linked to the thread where the review took place.

Where an editor finds a page that has passed General review but feels the coverage dissatisfying, they may reiniate the process: start a thread, leave a user talk note, and attempt to improve the content. If this fails, the template may be removed.

Currently there #### that have passed General review. There is no canonical list. Ideally, every article on Wikipedia will one day be able to pass General review.

Moved to talk to make way for general workhop.

Name

Per all of the talking here I've moved this to a more general title. Content review/workshop might be quibbled over but I think it sufficiently generalized. Marskell 13:03, 2 October 2007 (UTC)

Fuck, my fuckin' head is totally spinning.............aaaaaaaaaaaaaaaaaaaaarrghhhhhhhhhhhhh
But seriously, I'll try to add some ideas to the page here.cheers, Casliber (talk · contribs) 14:09, 3 October 2007 (UTC)

PR cons

To this I'd add: "Participation is so low many submitted articles don't get any review. This has caused many editors to abandon PR altogether, causing some to use FAC as a substitute.". As currently written, the PR con list hints at this but doesn't actually say it.Rlevse 13:31, 4 October 2007 (UTC)

Well, last time I looked, this is wikipedia, and "everyone can edit", so why don't you add it to the list? Dr. Cash 05:03, 5 October 2007 (UTC)

Approach

After some discussions with Marskell (visible on our respective talk pages, if anyone is interested), I would like to try adding some additional material to the top of the workshop page, with the goal of laying out a roadmap that tries to avoid some of the problems FA/GA/PR discussions have had. I hope it will be self-explanatory what I'm trying to do. A brief summary is that I'd like to start by getting folks to agree on goals and scope, and then work from there to identifying specific improvements that can be made. There are some risks in this approach -- for example it's easy to make the scope too big, and easy to try to come up with ideas that will fix everything in one go. However, if we can avoid those pitfalls, I think the benefit of working from goals first is that we might get some common understanding of what we're trying to accomplish, and build some trust among the participants in order to avoid the sense of "drive-by commentary".

I also am not particularly keen to advertise this page widely until we have some evidence that it's going to be useful. These are contentious topics; I know we already have some habitues of GA and FA here, and if we see things starting to gel I suggest we post some notices to attract more attention at that point. Right now, I feel like this is a workshop which needs to work, rather than be examined for effectiveness. Obviously we should be glad of anybody who stops by and contributes, but I think we might be better off for a few days at least if we see what progress we can make before asking for validation. Mike Christie (talk) 21:48, 4 October 2007 (UTC)

Query on recent edit

Malleus Fatuarum just changed "Produce well-written articles with accurate, complete, and neutral content" to ". . . broad, and neutral content", with the edit summary "Scope creep". Could we get a bit more explanation? I'm not sure I disagree with the change, but I don't really understand the edit summary. Mike Christie (talk) 13:21, 5 October 2007 (UTC)

I meant that the goal posts for GA seem to be changing. WP:WIAGA says that good articles are broad in their coverage, not complete, or comprehensive, as required for FA. So the scope of GA seemed to be overlapping with FA, hence scope creep. --Malleus Fatuarum 14:04, 5 October 2007 (UTC)

Just gonna free form riff on some ideas here:

So here's some ideas that are running through my head (after one beer to clear up the cobwebs):

  • A centralized review page, which will supplant GA. It will basically BE the GA page, and act as the gate-keeper for the FA page. Essentially, a single reviewer will look at the page, compare it to WP:WIAGA standards, and make one of three decisions:
    • Article is below GA standard: Reviewer leaves a detailed review on the talk page, listing fixes needed to make a GA. They needed list EVERY grammar error, nor every single uncited sentance, but ideally they should leave enough information that those that are taking care of the article can fix it.
    • Article is at GA standard but below FA standard: Reviewer adds the article to the GA list, but also lists places where they believe the article will need to be fixed to reach FA standard, as above.
    • Article is likely FA standard: Reviewer adds the article to both the GA list AND the FAC page, with a link to the user that originally nominated the article (The original nominator is expected to shepherd the article through the entire review process and see that requested fixes are made themselves). FA continues as normal.
  • If we do anything with Peer Review, we should somehow combine it with the GAR/FAR/FARC processes. A sort of centralized open comment page, perhaps with several sections:
    • Requests for general review
    • Requests for reassessment for articles that may be over-ranked (optional for GA, required for FA)
    • If an article fails GA but the original nominator thinks it should have passed, they should bring it for general review.

Upsides of this process:

  • Reduction of WP:SNOW-bad nominations at FAC
  • Largely keeps the same structure we have had so far
  • Gives PR a raison d'etre, which it seems to lack
  • Organized and orderly, and every process has a clearly defined role in improving articles.

Downsides of this process:

  • Cultural inertia (the FA regulars are sure to resent the GA as the gatekeeper for its process)
  • Extra levels of beurocracy and process
  • A backlog at one place can bottleneck the process
  • May be too complicated and may discourage people from nominating at all (unless we leave the grunt work to the reviewers... The nominators role should be the initial request for the review, and to make any requested fixes. The rest should be up to the reviewers).

Any comments?--Jayron32|talk|contribs 01:48, 6 October 2007 (UTC)

"A centralized review page, which will supplant GA. It will basically BE the GA page, and act as the gate-keeper for the FA page." I think the proposal fell at the first hurdle. --Malleus Fatuarum 01:58, 6 October 2007 (UTC)
Care to elaborate?--Jayron32|talk|contribs 02:48, 6 October 2007 (UTC)
You said it yourself: "the FA regulars are sure to resent the GA as the gatekeeper for its process". Surely where the GA process scores over the FA process is in encouraging editors to strive for a standard that isn't perceived as being beyond the reach of mere mortals? That's got nothing to do with being any kind of gatekeeper for FA. If FA wants to make FAC nomination contingent on an article already having achieved GA, that's a matter for FA, doesn't mean that anything has to be changed here. --Malleus Fatuarum 13:55, 6 October 2007 (UTC)
This seems like it would add far to much bureaucracy to the review process as a whole, and would just confuse things. I'm also not real crazy about the "gatekeeper" idea, either -- I don't think we want to depend too much one any one particular volunteer, and the process should spread the administrative burden a bit wider. If one person is "in charge" of it, and that person leaves (voluntarily or involuntarily), the process is broken until that person is replaced. We also don't want too many "Rauls" running around (part of FAs problem is that Raul overall has too much responsibility and power, and it would help to spread that out a little bit). Dr. Cash 02:29, 6 October 2007 (UTC)
You completely misunderstood. The entire process is still anti-beurocratic. By a single editor, I meant that the "gatekeeper" page will work EXACTLY, and IN EVERY WAY as GA works now, in the sense that only one editor does EACH review (not one editor doing ALL the reviews). Its not a Raul-like pre-FA czar, it just a statement of how GA works now: A single editor reads an article, compares it to WP:WIAGA and decides to promote or fail, it's a WP:BOLD process rather than a WP:CONSENSUS process. It is a way to integrate the GA into a more unified review process.--Jayron32|talk|contribs 02:48, 6 October 2007 (UTC)
Ok thanks for clarifying that. But now, your idea sounds even dumber than before, and just as bureaucratic and confusing. Dr. Cash 02:53, 6 October 2007 (UTC)

It seems to me that Jayron's suggestion, reduced to its essense, is rather simple. In fact it is just one addition to current process: if a GA reviewer is particularly impressed by an article they are reviewing, then they should be encouraged to nominate it at FAC after completing their review. That doesn't sound dumb to me, and it certainly isn't bureaucratic. Two benefits: nomination by reviewer helps to break down the adversarial "reviewers vs nominators" system; it may help to generate some goodwill if GA feeds FAC with its best articles.

However, forget all talk of "gatekeepers": that aspect of the idea won't work at all. Geometry guy 14:22, 6 October 2007 (UTC)

If that's what the idea amounts to, then I doubt that it would have very much practical impact. I don't know what the figures are, perhaps someone does, but I doubt that many articles get through a GA review without requiring some attention even to get to that standard. But more importantly, I think that a successful FA nomination (or a GA nomination for that matter) requires the commitment of at least one editor knowledgeable about the subject matter, to deal with the inevitable issues raised during the review. So whoever nominates the article really ought to be one of those prepared to see the process through, not some disinterested GA reviewer. --Malleus Fatuarum 15:17, 6 October 2007 (UTC)
Actually, part of my point is that the nominator would be disinterested. One of the problems with FAC, in my view, is that it is too adversarial: the nominator is passionate about the article, and other reviewers effectively say "if you want your article to be FA, dance for me". The focus should really be on the article, i.e., "this article is close to FA standard, lets see if we can get it there". Geometry guy 16:41, 6 October 2007 (UTC)
I think you give the game away when you say " ... let's see if we can get it there". FA reviewers don't work on the articles, they offer criticism of them; I think I remember one explaining that recently by saying that his "footprint" was bigger that way. It's an inherently adversarial process that requires a committed editor to see it through. Without structural changes to the way FA works, this proposal would simply lead to orphaned articles nominated by disinterested GA reviewers languishing until the FA director thinks that enough time has passed before failing them because the issues raised during the review hadn't been addressed. --Malleus Fatuarum 16:55, 6 October 2007 (UTC)
"...if you want your article to be FA, dance for me" is rather odd. (I thought I was the one trading in histrionics?) Jay's ideas aren't entirely without merit. Perhaps the reviewer could compare to the WIAFA page and add to GA if it's close, noting "recommend FAC." Marskell 18:50, 6 October 2007 (UTC)
Would you also be expecting the nominating GA reviewer to deal with the issues that would come up in the FA review? It seems to me that perhaps one of the things that GA and FA have in common is that both review processes rely on the participation of committed editors. Which a disinterested GA reviewer would very likely not be. --Malleus Fatuarum 19:06, 6 October 2007 (UTC)

I can do histrionics too - hey, it's fun! No, I would not be expecting nominators to deal with issues. I really would like to see a change in culture at review processes. I think reviewers could be a lot more proactive than they currently are (here I should applaud the many reviewers who fix problems with articles). I like Marskell's "recommend FAC". It is amazing how a little idea can make a big impact. Geometry guy 19:29, 6 October 2007 (UTC)

If your proposal has now become that reviewers should roll their sleeves up and get stuck into improving the article instead of being content to simply criticise it, then I think that already happens to a significant degree with the GA process; the culture of FA is quite different though. But the point you're not addressing is the need for a committed (and did I say knowledgeable yet?) editor to be involved. --Malleus Fatuarum 19:51, 6 October 2007 (UTC)
People roll up their sleeves at both. The question is volume of reviewers—that's an issue at every process and has no perfect answer. Marskell 20:04, 6 October 2007 (UTC)
People roll up their sleeves in the company of a committed editor who is knowledgeable about the subject. Even with that committed editor, the review process too often degenerates into an exercise in English grammar. But I've stated my point, I've got nothing else to add to what I've already said. --Malleus Fatuarum 20:22, 6 October 2007 (UTC)
What about two streams?
Assuming it's not rejected, the single reviewer says "Good short article" (too short to be FA—end of the line) or "recommend FAC". The latter would essentially be a queue. Perhaps PR becomes involved then. Marskell 19:46, 6 October 2007 (UTC)

I want to be clear that I am not wedded to any idea I spelled out above. From my point of view, there is one major problem with the way articles are reviewed at Wikipedia, and that is the disjointed nature of the review system. Maybe I can refocus this by explaining the problems my above proposal was trying to solve:

  • Problem 1) There are no less than 4 review processes that are at work at Wikipedia, and there is little coordination between them. FA, GA, PR, and the stub/start/B/A system are all handled in different ways by different groups of editors and serve different purposes. Wikipedia should have a centralized review system, the goal of which is to provide critical review that causes editors to improve articles to FA status.
  • Problem 2) Peer Review is WAY underutilized. Anyone who has spent enough time around this process is familiar with the dreaded FA-PR loop of death. An FAC fails with numerous notes to complete a Peer Review before the next nomination. The Peer Review languishes for a month receiving little or no attention. The article is renominated, unfixed at FA. Rinse and repeat.
  • Problem 3) There is a wide difference of opinion on the purpose of each of these processes. Some people nominate their articles for a "rubber stamp" endorsement of their work. We have all seen it where a nominator exhausts huge amounts of energy trying to explain why they shouldn't have to provide inline cites, neededn't clean up their prose, or explain away EVERY objection to their article with "the page you say I should follow is only a guideline, so I should be able to ignore it" or the like. Even between the processes, there is a lot of animosity. Many editors feel that FA is arbitrarily tied to the more esoteric and technical aspect of the MOS (We have all seen FAC's being opposed for having a misused endash or an improperly wikified date), while many other editors feel that GA is a redundant process or that it is to heavily dependant on the opinion of single editors. Likewise, the project level review system can be inconsistantly applied between projects; what one project calls A-class may be B-class in another project. If we had a single reviewing system, with a single set of standards, it would reduce a LOT of these problems.
  • Problem 4) What I call "cultural inertia" above. In a perfect world, we would devise a new process that replaces everything we have, it would be logical and simple and work well. Even if we take for granted that such a process exists, there is often so much emotional baggage in the processes now that even if EVERYONE AGREES the current system needs fixing, NO ONE CAN AGREE that their part of the system should be scrapped or modified for the new system. Unless we can overcome this last part, this whole idea is doomed from the start.

There it is... Any ideas? --Jayron32|talk|contribs 23:57, 6 October 2007 (UTC)

If all these processes are in their own way improving the standard of articles on wikipedia, which I've got no reason to doubt that they are, then I don't see the problem. I really can't see any reason to replace the present matrix system of assessment with a heirarchical one. In fact I can even see some advantages in the present matrix system. FA is different from the rest only insofar as it's publicly recognised by a little star in the top right corner of an article; but so what? --Malleus Fatuarum 01:09, 7 October 2007 (UTC)
That's a fair question. I will return it with another question; does the current system work well enough or could it work better as a more organized system? I mean, the current system works to a point, but could it work better if it were better organized? Is good enough, well, good enough or can we do better? --Jayron32|talk|contribs 02:52, 7 October 2007 (UTC)
Anything can be done better. You ask: "does the current system work well enough ...", the answer to which must be "who knows"? What does "better" mean? What are the goals of the current system? Which of the current processes leads to the greatest improvement in the quality of articles on wikipedia? How would the success of any new process be measured? Fundamentally, I believe that if you can't in some way quantify it, then you can't improve it. So I would suggest that the first thing to do would be to set up some kind of metrics programme for whatever it is that you feel needs to be improved, instead of fiddling while Rome's perhaps beginning to smoulder. --Malleus Fatuarum 03:20, 7 October 2007 (UTC)
I'm not sure that spending inordinate ammounts of time analyzing the situation is in order. There is the problem of paralysis by analysis: Processes that obviously need fixing by simple observation go unfixed while we try to prove what we already know: the current system, while it works, works poorly and can be replaced by a better system that would work better. This is hypothetical; I am not saying the system is broken, but what I am saying is that to deny that personal experiences with the system are an invalid way of measureing the system is simply paralysis by analysis... There will never be enough data in a timely enough manner to act upon, so it will simply kill needed improvements. What is for sure is that a) many people outside of these processes resent them and b) some of them, ESPECIALLY peer review, DO NOT WORK. It doesn't take much analysis. Look at the current peer review page, and count how many responses each posting gets. There are currently 103 open peer review requests; and by my count only 47 have received any comments. It is not uncommon for an article to languish there. There is also the non-quantifiable problem of respect for the entire process. How do you measure how much respect the review system gets among editors. GA gets a fair amount of undeserved shit because of the perception that it is unchecked (not true) biased (not true) and redundant with FA (also not true). Don't imply that something cannot be fixed unless a number can be placed next to it and we must somehow improve that number. Sometimes statistics are useful, othertimes they are damned lies. Lets not kill the entire improvement process before it starts because we don't have some meaningless numbers to improve on. I will bold this so it can be clear: I am not proposing that a single one of my suggested improvements be acted upon However, you seem to be implying that even small improvements or tweaks to the system cannot be made without some long, detailed statistical analysis, which to me seems like stonewalling. --Jayron32|talk|contribs 05:02, 7 October 2007 (UTC)

Well, I have to agree with Jayron here about one thing; the issues with the review process as a whole on wikipedia has some quite fundamental issues that are not easily fixed, and not localized to be able to clearly point the finger at either FAC, GAN, or PR. These issues aren't going to be fixed overnight, and we can't really solve them by tweaking a couple of templates and review instructions, as we've done quite a bit recently with improving the GA system this past summer. The changes are going to take some time, and probably won't be immediately evident. I would suggest that we should take a step back and chill a little bit; let the GA sweeps conclude and maybe do another backlog elimination drive, and continue to promote FAC and develop it. These seem to be our two strongest review programs going right now. At some point, we'll figure out how PR fits into the whole equation. But it will take time. Dr. Cash 06:18, 7 October 2007 (UTC)

Derek.cashman is right. My tone was becoming combative, and I unequivocally apologize for that. However, lets not dismiss the entire idea of a centralized review process before we see what can be hammered out here and if we can generate some bold and new ideas on what might improve the process.--Jayron32|talk|contribs 06:23, 7 October 2007 (UTC)
That there's inefficiency is understood, which is why this workshop exists. I agree PR stands out most especially; I think it should be handed to the projects.
So it doesn't get lost in the shuffle, would two streams work for GA?: one to identify good short articles, and one "recommend FAC" queue. Marskell 06:24, 7 October 2007 (UTC)
I haveto say I thought things were working ok in all areas apart PR, though PR's inaction was havinga knockdown effect on the others (not hugely but noticeable). To me this whole thread, though interesting, is investing a huge amount of input for very little payoff. Given it is a volunteer project the maximum benefit I can see is only going to be a minor streamlining really. Why not just concentrate on bolstering PR?cheers, Casliber (talk · contribs) 06:28, 7 October 2007 (UTC)
The problem is that suggestion has been tried before, and without many results. In many ways, I would be tempted to eliminate PR completely. Titoxd(?!? - cool stuff) 06:46, 7 October 2007 (UTC)
I don't think it really fits to combine "short" with "good" articles (e.g. "good short articles"). Remember, we have to vastly different things here; quality and quantity. There's lots of GAs that are not short, and not necessarily all of them will be promoted to FA. But FAs don't necessarily have to all consist of ridiculously long articles, either (granted, an article as short as a stub will never be promoted, but we have to get away from thinking that just because an article is short, it either can't be FA or has to be GA). Dr. Cash 07:54, 7 October 2007 (UTC)
I'm getting so many mixed signals on this. In the edit history of this page, Lara says that identifying good short articles is "precisely what GA does." But every time I suggest explicitly focusing on short articles people say "no, that's not really the target." Dunno. Marskell 10:29, 8 October 2007 (UTC)

Idea - devolving PR to wikiprojects

Howsabout trying to devolve PR to the various wikiprojects? Serves to promote importance of wikiprojects by hopefully encouraging article improvement there rather than on a general page, with a PR coordiantor who can leave PR to only contain articles not related to a psecific wikiproject -the reduction in the queue might make for more reviews there. cheers, Casliber (talk · contribs) 06:54, 7 October 2007 (UTC)

In my view, wikiprojects are perennially "mentioned", to the point where a newcomer might get the impression that they are a fundamental organizational component of wikipedia—but really, on the whole, they barely exist, except in name (in my experience). I don't think PR could be channeled to projects for that reason. ¶ Since I'm on this page: I do like this general movement towards combining peer review and GA, and simply letting the nominator have the choice of saying, "please review this" or "please review this and add it to the list of good articles if you think it passes". I don't see how it couldn't be an improvement, with the only hurdle being the community that doesn't want GA to be touched. (And incidentally, I generally like GA, and I don't view it as duplication or inefficient.) –Outriggr § 08:27, 7 October 2007 (UTC)
There already exist several topic-specific peer review processes, as well as Wikipedia:Scientific peer review, which appears to be marginally more successful that PR itself. Some WikiProjects are alive and well, and indeed very active. Others need encouragement. Geometry guy 10:34, 7 October 2007 (UTC)
That's a point. Maybe if PR just ceases to exist then it will all go to GA anyway. cheers, Casliber (talk · contribs) 09:58, 7 October 2007 (UTC)
Couple of comments: One is that I value 'vanilla' Peer Review because it gives me access (sometimes :)) to people from different areas. I can always ask for comments at my local WikiProject. Devolving PR to WikiProjects could lead to articles becoming 'specialist' rather than 'generalist'. That is, they would be reviewed by people who are familiar with all the jargon and concepts involved in the topic and would be less exposed to new thinking on the topic. Second one is to repeat the point made above: Not all Wikiprojects have the organisation or resources of MilHist. Some barely exist other than in name. Cheers. 4u1e 16:44, 12 October 2007 (UTC)

Development of the above

Although I rather like the idea of a centralised review process for the entire WP project, and we may even manage to hammer out something that PR, GA and FA will all agree on (stranger things have happened), as mentioned above eventually these 'streamlining' proposals will also need to take account of almost every wikiproject out there. MilHist, for example, has its own A-class review process; IMO correctly since the best judges of article content quality are likely to be found working in their own areas. It would also make sense to me to involve the LoCE early on.

Concerning the overall review process, this might lead to a three-tier system, where GA (or its equivalent) reviews articles for MoS/prose/policy compliance; A-class (or equivalent) where content reviews are completed in-project by subject experts, and finally FA (or equivalent) where the whole package is quality-checked and made as good as it can possibly be. The LoCE could be formally brought into the process at some point (maybe during the first stage). Compared to the current set-up, this would more clearly define and demarcate the roles for each existing area, would ensure that articles reaching the FA stage are already very good, and would place the burden of initial contact on the GA project which, of the candidates, seems best able to cope with an increasing workload. I can see loads of potential problems with this (not the least of which is the role of PR), but since we're just brainstorming... ;) EyeSereneTALK 16:07, 7 October 2007 (UTC)

I do like the idea of bringing in copy editors at some point in the review process. But I wouldn't think we want this at the GAN stage, rather at the FAC stage. One of the biggest things I hate about FAC is the tendency of reviewers to primarily cite a bunch of WP:MOS and copy editing types of suggestions, with little guidance on improving the actual content. I would much rather have the FAC reviews focus more on the actual content, overall readability (beyond simple grammar), and citation issues, and then just simply fix any major copyediting issues in the end automatically. So maybe the FAC closing procedure needs to change so that once the article's discussion is closed, the article is placed into a brief queue whereby it is gone over by an editor with extensive experience in copyediting, for a final check of the minor details. Once it passes that stage, it becomes WP:FAC.
Of course, we'd need more than just Raul involved, since I don't think he wants to check every single article that passes FAC himself. Maybe have 5-10 assistant FA directors, or certified FA copyeditors, which would do the final check and certify it. Raul could then promote all the articles that they certify. Dr. Cash 20:02, 7 October 2007 (UTC)
I see the idea of actually scrapping PR has been floated... Perhaps the page could be a list of targets that do work: See scientific peer review, MiltHist review, etc. I'd also like to encourage user talk: "editors are strongly encouraged to seek out other users for a review either through the history of similar articles or at Wikiprojects." We could have a list of people willing to do reviews for various topic areas on the PR page and the actual reviews could be done at article talk. That would be far more useful than simply throwing up your review and hoping somebody stops by. Marskell 10:29, 8 October 2007 (UTC)

PR proposal

Peer review

The Wikipedia Peer review is a centralized list of active review processes and users willing to help with reviewing. Reviews themselves are not listed here as a generalized peer review has proven uneven in producing content improvement. Rather, editors are either directed to a more specific review process or provided a list of users who are willing to be contacted on their user talk pages. In the latter case, reviews take place on article talk: start a thread and provide a link to the volunteer editor(s). If a subject area is not listed here, editors are strongly encouraged to seek out other users for a review either through the history of similar articles or at Wikiprojects. If there is a topic area that you are interested in, you may volunteer as a reviewer here.

Marskell 13:18, 8 October 2007 (UTC)

I'm not sure I like the tone of the above. Check below for an alternate idea. Plus, PR is a Good Thing (tm), but it needs fixing so it works better, not scrapping.--Jayron32|talk|contribs 16:36, 8 October 2007 (UTC)

What's the word for an article that's good enough?

I think it would be good to have a way to refer to an article that is "good enough". What I mean by that is an article which, if we were a for-profit encyclopedia, we would regard as good enough to publish. "Finished" would be the right word if it weren't for the fact that we know many articles will continue to evolve. Perhaps "!finished", to use a standard Wikipedia approach, though that's a bit in-group-ish. Anybody got a better idea?

The reason I want this term is that I want a way to talk about articles that are good enough to accept, without referring to the existing processes. Featured articles are intended to be the best of Wikipedia, but lists, pictures, portals and topics are done by independent processes. I also don't want to bias discussions by making it appear that I'm talking about FA as the ideal process. Of course, if we (and FA, and Raul) were to agree that featured articles and !finished articles should be one and the same, that would be an important point to establish. But until we've agreed that I'd like a word we can use.

Then maybe it would be a good idea to define what "!finished" means -- we could do that in terms of FA/FL/FP/FT/FPO etc if we like, and could then figure out what's missing from that definition. It might not be possible to go into every nook and cranny (when is a mainspace template !finished? a mainspace category?) but it would be good to at least outline what's there. It's often been said that short articles aren't well addressed by the existing content review processes. I think that if short articles can be !finished, then they should be addressable; if not, they shouldn't exist. Mike Christie (talk) 20:30, 8 October 2007 (UTC)

I'm afraid that I really can't see how altering the accepted dictionary definition of "finished" would be likely to lead to anything other than further confusion. Fundamentally, every article is "!finished", and always will be. --Malleus Fatuarum 21:11, 8 October 2007 (UTC)
All I meant to do with "!finished" was to draw an analogy with "!vote" which looks like a vote, but isn't. The word isn't important, but I think the concept is. Is there an existing term in use in WP talk that means this? Mike Christie (talk) 21:26, 8 October 2007 (UTC)
I think that there is; well not one word admittedly, but two. "Good Article". --Malleus Fatuarum 21:34, 8 October 2007 (UTC)
"Acceptable". Geometry guy 21:37, 8 October 2007 (UTC)
"Good Article" is problematic because if it refers to "What is a good article?" it implies that FA is asking for higher standards than we actually need to have. I don't think that would have consensus support; FA is intended to identify articles that are examples of what Wikipedia should strive towards. (I am not asserting it achieves that, just that that's the goal.) GA is intended to have slightly weaker criteria than FA, so that would imply FA is unnecessary, and that's not what I'm looking for. The problem with "acceptable" is that it sounds like it meets some low minimum.
I want a word that we can use in this workshop. For example, if the word were "publishable", it would be used in contexts like this: "If an article complies with everything in MOS, does that guarantee the layout is of publishable quality?" It seems to me that the goal of all the article writing processes is to make the articles better; if we don't have a word for the direction we're moving, then we can't agree on how to get there. What we have now is some subsidiary targets such as FA, GA and so on.
We could also just use "better", but then the target would be "good enough", as in "good enough for a consensus to be reached that the article can be published as it is". I was hoping for a specific word. If there isn't one, it's not that important, but if someone comes up with a good shorthand for this I'd like to see it come into use. Mike Christie (talk) 21:56, 8 October 2007 (UTC)
Whatever word you pick, it has to be defined. What is the minimum standard for a wikipedia article, and what are the criteria by which it's judged? For what it's worth, I think that the GA standard came pretty close, which is why I've not been happy to see some recent moves that have potentially made that minimum standard more and more onerous for editors to reach. FA may be reaching for the stars, but at least GA was trying to get editors to raise their eyes from the gutter, which is at least a start. --Malleus Fatuarum 22:11, 8 October 2007 (UTC)

The word acceptable is rather subtle, which was why I mentioned it with no further comment. I agree that it suggests a minimum standard, but what this minimum is depends entirely on what is accepted. At WP, the obvious definition of acceptable would seem to be: meets WP:NPOV, WP:V and WP:NOR. But if you really interpret these policies rigorously, then you actually get surprisingly close to requiring featured article standard: the main things missing are house conventions (WP:MoS). Most articles on WP are not acceptable. We admit them anyway, because it is usually better to have an unacceptable article than no article at all. And we publish them. Many encyclopedias publish articles which are not particularly good, but they are usually acceptable. Geometry guy 22:44, 8 October 2007 (UTC)

I don't think the most helpful thing we could be doing to improve the process is making changes in the name based on subtleties of interpretation. I don't see anything wrong with keeping it as good articles. Is there something wrong with the concept of requiring articles to be good? No. Good and featured do not even come close to meaning the same thing. VanTucky Talk 22:56, 8 October 2007 (UTC)
No one is proposing changing any names. I agree that good and featured have very different standards. Mike's purpose, as I understand it, is simply to avoid divisive discussion by using some other terminology than the existing processes use. Geometry guy 23:08, 8 October 2007 (UTC)
Yes, that's what I intended. However, we can use circumlocutions when we need to, so it doesn't matter too much. Mike Christie (talk) 23:52, 8 October 2007 (UTC)
I think it matters a great deal, unless you're an advocate of the Humpty Dumpty school of debate. --Malleus Fatuarum 01:49, 9 October 2007 (UTC)
No article below "Featured" status should be "acceptable". The goal for EVERY article should be to reach featured status; it indicates relative completeness in terms of representing present scholarship on a subject, briliant writing, excellent referencing. If there is not enough information in reliable sources to reach feature status, chances are there aren't enough sources for a full article anyways. The concept of "good enough" encourages complacency and mediocrity. Every article should strive to be the best it can be, not simply "passable"... --Jayron32|talk|contribs 04:18, 9 October 2007 (UTC)
"Publishable" might actually imply a standard above FA. Material for publication should be vetted, for examples, for consistent Brit or American usage, for consistent J. Smith or Smith, J., for consistent full stop placement, for consistent dash usage. And so on. In practice, FA lets some of this slip through and it actually rankles people to be asked to fix their dashes. I think very few of our larger articles are actually fit for publication in these terms.
Mike's basic idea is sound though, particularly for smaller articles (hence the General review idea above). We need a short and sweet way to say "yes, this six K of prose is what's warranted for this topic." Marskell 10:54, 9 October 2007 (UTC)
The term "publishable" is not very accurate to use here, as different publications have their own standards. Furthermore, with the world wide web out now, the term "publishing" does not just refer to paper-based documents anymore, but also to websites (though many people, including publishers themselves, haven't realized this yet!). Essentially, every one of the 2 million + articles on the english wikipedia has been "published" already, even though they're in a constant state of flux and constantly changing. It would be good, however, if wikipedia were to better enforce some of its grammatical and organizational standards better, so that our best articles had a consistent look-and-feel. Maybe adding an extra copyediting step immediately after FAC approval to ensure that the article meets the manual of style. The would also help to cut down on the really lame MOS comments and opposes during the FAC stage, allowing editors to focus more on the content. Dr. Cash 05:40, 11 October 2007 (UTC)
Actually, in my experience, most of FAC ends up being a bunch of MOS nitpicks. MOSDASH is a favorite target for most articles, also MOSDATE shows up a lot. FAC ends up being pretty throrough; it is not often that an article will make it through with any copyediting issues. --Jayron32|talk|contribs 05:43, 11 October 2007 (UTC)
Responding to Mike Christie's proposal above, I think what you have described is the current "B" category -- & what I had once envisioned the "GA" category to mean. Unfortunately, there is a tendency to rate articles as "B" more on length than how well they cover the topic; if one can only write, say, only 1,500 words on a given subject due to limits of material (e.g., to say more either means padding out a plot summary or discussing various points of view to an unnecessary degree that borders on original research), someone who doesn't know the subject will stubbornly insist that it is a "Start", & refuse to listen to any reason that it should be rated as a "B" or higher. On the other hand, if we attempt to come up with a formal process to determine what makes a "B" article, we'll end up with reinventing the FA & GA processes with all of their drawbacks but none of their advantages. -- llywrch 17:23, 12 October 2007 (UTC)
I can agree with many of the above! What shade of grey do you want? For the 1.0 project, which has published some articles offline (and we plan to do a lot more), Llywrch is right - B-Class tends to be the minimum quality standard for most articles (though we include a few decent Starts such as Martinique to ensure completeness and coverage in some areas, e.g. many African topics). If you actually want to publish a set of articles from what we have now, you have no choice but to use lots of Bs. (It's also true that with B-Class an article may be rated as much on sheer length as for its actual quality.) I would say that I think a modern-day GA is normally a pretty high standard, and if we could have every article in a release be GA I would be thrilled. Of course FA is a better goal towards which all should strive, though IMHO a quality version FA (FA+?) would be much more suitable as a gold standard than the present flawed FA process. That would bring together MOS and semi-brilliant prose with rigorous fact-checking - but we will probably only have a handful reaching that level! Walkerma 03:56, 13 October 2007 (UTC)
"I would say that I think a modern-day GA is normally a pretty high standard, and if we could have every article in a release be GA I would be thrilled. Hear, hear. FA is of course the gold standard, but there has to be something credible between it and a stub. --Malleus Fatuarum 04:06, 13 October 2007 (UTC)

How this group can be most productive

Marskell suggested, above, that we might want to consider focusing on peer review, since there have been several comments to the effect that it needs a rethink. I think this is a fine idea, but I wanted to preface it with a question about how this page can best work for the benefit of the other content review processes. Obviously we need people here who understand those processes, but we don't want to duplicate the activity on the various talk pages such as WT:PR, WT:FAC and WT:GAC. This group is valuable mainly if there is some synergy or cross-fertilization to be gained from having GA, FA, PR, MILHIST (and so on) people coming to a consensus about their different understandings of these processes.

So I think there are some questions we should try to have answers to.

  • How widely should we advertise this page? It's not a secret, of course, but is it the right thing to do to post a note at PR, FA, GA and so on? I think it might be time to do this, with some caveat such as I give above about this page not intending to duplicate all the other talk pages.
  • What's the process for taking an idea from this page and trying to get broader consensus? Do we want to make a choice to periodically focus on specific problems, rather in the manner of the Collaboration of the Week that some WikiProjects do? If so we could focus on PR next. We could also have a very simple process along the lines of nominating ideas/problems on this project page for the next "problem focus of the week" and picking one each week to discuss.

-- Mike Christie (talk) 03:20, 11 October 2007 (UTC)

There are ways to get listed across the project; probably the most important is getting on the Centralized Discussion template. Also, posing in the various Village Pump pages is a good way to advertise. See: Wikipedia:Centralized discussion, WP:VPP, WP:VPM, and WP:VPR for the best ways to advertise across the project. Also, there is a section in RFC for inviting further discussion, IIRC... --Jayron32|talk|contribs 05:38, 11 October 2007 (UTC)
Concerning concentrating on PR, is this prematurely narrowing our review? It seems to me that there is no clear consensus yet on either the eventual outcome we wish this review to achieve, or the specifics of how we plan to get there. Are we just after more efficient and clearly-defined incarnations of the existing review processes, or are we looking at a total, ground-up restructuring?. If the second, it would make sense to define the new process and the roles required within it before looking at the existing review processes to see which roles they might take on. EyeSereneTALK 11:03, 11 October 2007 (UTC)
The converse problem Eye, is that if we don't narrow in on something we may be diffusely talking about everything at once and achieving nothing. I mentioned PR because it seems to be the one process that everyone agrees needs help—good in principle, poor in practice.
Re initial, a problem of the week isn't a bad idea, though we should let it run longer if there is still a buzz. I think after we identify a problem, we go to the talk of the process in question and gather a few other people. After we have come up with workable solutions we advertise it across the site to gather fuller consensus. Let's start. Marskell 12:27, 11 October 2007 (UTC)
If others agree, let's write a paragraph on the workshop page saying that's our process, and create a section for nominating problem of the week. I agree about letting it run if it's going well. Mike Christie (talk) 12:30, 11 October 2007 (UTC)
I'm not convinced that this isn't putting the cart before the horse, but I do accept the logic of Marskell's position. We don't need yet another talking shop ;) However, should there end up being a major re-organisation of the whole review process, we may need to accept that this approach will lead to roles being defined without having any clear end-point in view... which may result in them being redefined again later as the big picture develops ("Didn't we just discuss this; I thought it was settled?"). </cassandra mode> We'll know nothing if we don't try, though, so why not go for it and see how it works. EyeSereneTALK 14:28, 11 October 2007 (UTC)
Well, there are two parallel processes that need to be done, and I am not convinced that they aren't BOTH worthwhile:
  1. Fixing individual review streams (PR, FA, GA, LOCE, A/B/Stub/Start, etc.) where they need it.
  2. Establishing a Wikipedia-wide, unified review system that reduces friction and increases positive and helpful reviews for improving articles REGARDLESS of their current quality.
There is no reason that both cannot occur simultaneously, or nearly so. --Jayron32|talk|contribs 16:35, 11 October 2007 (UTC)

Suggest adding:


Marskell 17:55, 11 October 2007 (UTC)

I quite like this text (and the associated idea), but I also tend to be in favour of notifying individuals rather than entire projects: a personal message encourages someone to contribute, the contribution is more likely to be sustained and involved, rather than drive-by, and we can target people who understand (and have opinions about) not just an individual aspect of content review, but also its relation to the big story.
This page is already getting quite a bit of traffic and I think the core group should not get too large for the time being. At the moment, though, the group is mostly GA and FA, and we need people with strong interests in peer review and Wikipedia 1.0 to be part of the core. I'm not really involved in these processes, so I am no expert to comment, but two editors that spring to my mind are Awadewit and Walkerma. Geometry guy 18:50, 11 October 2007 (UTC)
I also like Marskell's text. Maybe change the last two sentences to "After a Current topic is decided on, the process in question may be notified if more participation is needed to workshop the idea. When possible solutions have been arrived at, the wider community will be notified through an appropriate announcement location, such as the individual process pages, or the Village Pump and Centralized discussion template for ideas with impact beyond a single process."
I like the idea of individual notifications, too. I will leave a note for Awadewit, since I've interacted with her briefly. I don't know Walkerma; Geometry Guy, would you leave him/her a note? Mike Christie (talk) 22:48, 11 October 2007 (UTC)
Will add: with Mike's tweaks, but without "[processes notified] for ideas with impact beyond a single process." This might be taken to mean that if a reform is limited to one project, we'll institute it unilaterally. (So I read it.) Per G'guy: "Involved editors are encouraged to directly inform others who participate in content review, where there are active discussions." 'Are encouraged' is a shift in register; I don't know how we can mandate informing individuals directly. (Though better utilizing User talk would certainly be good for content review in general.) Feel free to tweak my tweaks. Marskell 20:38, 12 October 2007 (UTC)
I've notified both Walkerma and Awadewit; Awadewit is busy but said she would try to participate; don't know about Walkerma yet but I agree with Geometry Guy that a WP 1.0 perspective would be valuable.
I made one more tweak to Marskell's addition, in order to avoid the implication of unilateral action. I think it's wise to be as careful as possible with the wording in that regard: we are an idea shop, and no final decisions on implementation can be made here. Mike Christie (talk) 22:14, 12 October 2007 (UTC)

Picking the Current Problem

One last process question before we pick a problem and start brainstorming: how do we come to consensus on the Current Problem? The brainstorming page's list is a source of candidate problems, though I think I'd generally like to see the problems phrased positively rather than negatively. For example, I'd prefer "How do we improve the relationship between GA and FA reviewers in order to focus on actually reviewing?" to the current wording, which is "Constant infighting between FA & GA reviewers takes time away from actually reviewing articles". (I'm not suggesting we pick that particular problem.)

Perhaps another front page section which lists problems? Then put the Current Problem at or near the top of the project page until consensus is reached that it's time to change it. Then a topic on the talk page to pick the next Current Problem: maybe straightforward straw polling/voting would be good enough. If we can agree on this, we can go ahead and start building that list of problems now, drawing from the brainstorming page if we agree with those statements (and, as I said, perhaps rephrasing them). Mike Christie (talk) 22:26, 12 October 2007 (UTC)

Well, in part, realizing that neither GA nor FA is going to go away would go a long way in improving relations. But maybe that's just me. Titoxd(?!? - cool stuff) 00:14, 13 October 2007 (UTC)
Not just you, that would be me as well. --Malleus Fatuarum 01:14, 13 October 2007 (UTC)
Since the proposal as I understand it seems to be to concentrate on specific issues rather than the macroscopic process, it makes sense to me to have a to-do list, even if it starts out as only a few items. It's most likely going to be evolving anyway; sorting out one problem will probably throw up another two ;) A straw poll would be suitable IMO for choosing the 'issue of the week', and I think Mike's point re phrasing is important. This must be a constructive process, not an accusative one. I only have one question: should there be a time limit on discussing one problem before moving on to the next, and if so, what? EyeSereneTALK 10:07, 13 October 2007 (UTC)

Query regarding goals of reviews

What is it that the community wants to achieve with these reviews? We cannot really go forward with any revisions, big or small, without coming to an agreement on what the goals of the reviews are. Awadewit | talk 02:30, 13 October 2007 (UTC)

That's an easy one, better articles. The underlying problem seems to be an unwillingness to come to an agreement on how much "better" is "better" enough to get to each stage, and the ongoing FA/GA dispute is damaging in that respect. --Malleus Fatuarum 03:40, 13 October 2007 (UTC)
Although Malleus is right, I think I'd be more specific and say that the goal is (or should be) to show what needs to be done to raise an article to the next level. Walkerma 04:19, 13 October 2007 (UTC)
I'd be even more specific and say that the goal is to do what needs to be done to raise an article to the next level. Not to sit in judgement over it, waiting for divine intervention. --Malleus Fatuarum 04:26, 13 October 2007 (UTC)
I don't think moving an article to the next level is so simple - not everyone agrees on what the "levels" are, for example. Also, after having gone through FAC twelve times now, my impression of that process is not that it is cooperative, but that it is evaluative. While GA is evaluating against a set of criteria, it tends to feel more cooperative to me. This is just my personal impression. I think that this may be one reason why FAC is gaining a bad reputation. I wish FAC were either a place where an article passed or failed as it stood or a place where reviewers and editors worked together to improve articles. (What is this FA/GA dispute of which you speak? I've seen passing indications of it, but I didn't realize it was an out-and-out war or anything.) Awadewit | talk 04:33, 13 October 2007 (UTC)
I try to make my reviews collaborative at FAC and get stuck in and help (if asked, or more recently even if not!) as Malleus is right - the aim is better articles not to sit in judgement of others. I used to list all errors, eg misspellings etc. now I try to fix uncontroversial glitches as I go as it saves a heck of alot of time. I realise some reviewers can be difficult at FAC but I have also seen the other where nominators rather than try to address issues raised get annoyed and/or uncooperative instead. cheers, Casliber (talk · contribs) 10:57, 13 October 2007 (UTC)
Yes and no... I like to be collaborative when I do such a review, but then again that's partly why I don't now do GA or FA reviews (only ad hoc peer reviews) - the fixing up takes a lot of time. Heck, I don't even find the time to fix up articles I'm trying to work on!. Even when I do a quick review for 1.0, I often add a photo from Commons or rewrite some terrible text - and then I often end up getting one article reviewed instead of ten. I think a "drive-by" review can still be very valuable if it's written as constructive criticism. If everyone takes the time to fix up every article we review there would be a much bigger backlog at GAN and FAC. If it's a lot of work I sometimes like to clean up one section thoroughly, to show what I mean, and let the main editor do the rest. In short - a thorough review with feedback is good, but a thorough review with cleanup and feedback is better. Walkerma 18:13, 13 October 2007 (UTC)
If everyone took the time, then there would be no backlog. I agree that a collaborative review takes time, and a significant commitment from both the reviewer(s) and the nominator, but what's the point of any other kind of review? "Drive-by" reviews in particular are very demotivating. --Malleus Fatuarum 18:46, 13 October 2007 (UTC)
The intent of the goals section of this project page was to capture the answers to Awadewit's question, above. Is there anything from the comments above that isn't listed there? Mike Christie (talk) 19:32, 13 October 2007 (UTC)

List of problems on front page

Per some of the discussion above, I've put a candidate list of problems on the front page. Please edit as necessary. I got this list by going through Dr.Cash's original list of problems that was moved to the brainstorming page, and rephrasing them per the description I gave at the top of the list. I think once we've got this list in reasonable shape we can pick a Current Problem, although there are other conversational threads recently started above we need to keep going as well. Mike Christie (talk) 14:04, 13 October 2007 (UTC)

Another idea for how to handle Peer Review

Change the locus of the review. The Peer Review page still exists, but no actual reviews happen on that page. Instead, it acts as a clearinghouse for articles requesting Peer Review. Reviewers come to the Peer Review page to see articles that are requesting review. Each article title will use the {{La}} template and that is it. The reviewers are directed to make comments on the article's talk page, perhaps under a designated Peer Review section, which can be set aside on the talk page itself, perhaps by a border and color change, so as to indicate it is the official "peer review section". That way, relevent improvement discussions are moved to the talk pages of the articles, where it is more appropriate and more likely to attract more attention; the PR page is shortend significantly (the page size is one hindrance of productive work there) and otherwise the process still exists. Ideas?--Jayron32|talk|contribs 16:36, 8 October 2007 (UTC)

Isn't that essentially what's posted in the above thread? Marskell 10:43, 9 October 2007 (UTC)
No, because the articles are still listed at PR. The above is essentially an historical tag telling people that since PR doesn't work, it has been trashed, and leaves them to go elsewhere without much guidance. --Jayron32|talk|contribs 03:13, 11 October 2007 (UTC)
But it's precisely the leaving it on PR that's failing to work. And the above wouldn't be a dead page at all: it would have lists of people willing to review particular subject matter. I'll paraphrase qpb from the original village pump thread: at the end of the day, if you want people to look at your page you need to go to user talk and ask them to look at it. Marskell 12:20, 11 October 2007 (UTC)
AH, so what you are saying is that PR will change from being a place where you go to "drop off" an article to be reviewed to being a place where you "pick up" a reviewer who would like to do the reviewing for you? Interesting idea. I think I like it. Sorry I didn't pick that up before. The text box left above spent so much time subtly (and not so subtly) trashing the old process that it was unclear what the new process was to be. Not sure it will be enacted, but I dig it. Its a neat idea. --Jayron32|talk|contribs 06:14, 12 October 2007 (UTC)

(←) I think this could work, but I agree with Jayron that the boilerplate wording should be tweaked. More of a notice to say "The PR process has changed" and then the new instructions. Also, if it's just a list of willing reviewers, perhaps break it down into categories, similar to the main categories at GA, so that reviewers can list themselves under the topics for which they are knowledgeable and interested. Lara❤Love 15:07, 15 October 2007 (UTC)

The text above was not meant to be placed immediately—we'd need transition wording introducing the change for about a month. Categories: yes, that would be the main idea. I think it should be much less over-specific than the GA list but slightly more specific than FA.Marskell 15:31, 15 October 2007 (UTC)
Ummm, color me outdated, probably. I was in PR a long time ago.. I created a "desk" and I was supposed to wait for someone to drop a request in my "desk". And I waited. And i waited. And i quit in disgust at the passive nature of the system. Still the same? is that what we're discussing? Then i don't like it. :-) --Ling.Nut 15:34, 15 October 2007 (UTC)
I don't know what you mean. Marskell 22:04, 15 October 2007 (UTC)

Optional team reviewing?

This something that LaraLove brought up to me as an option. Even the most experienced reviewers only have one pair of eyes, and can miss small (or pivotal) issues in an article through sheer accident. Just as no reputable news publication calls all done on a piece that only a single copyeditor has viewed, collaboration in reviewing might be an essential step in improving the quality of reviews. I think an optional way to review an article in two-man teams might be a good idea. Of course, implementation of this might best be started after we kill the current backlog. Probably the two problems to be dealt with are creating/updating templates for dual reviewing, and what to do in the case of a fundamental disagreement between reviewers. VanTucky Talk 22:21, 8 October 2007 (UTC)

Thanks for posting this, VanTucky. I envisioned a subpage of the project page where there'd be a brief overview of how it works. Basically, anyone wanting to try out doing tandem reviewing could list their name and their strong points (for example, I'm great at nit-picking prose, style issues, references and the like, so I'd want to pair up with someone who's good at finding broadness and comprehensiveness issues). It would need to be clearly outlined that any issues/disagreements between the two reviewers should be handled on their respective talk pages and not the article's talk page. It's almost like the second opinion option, but it's a set team of two on all the reviews of those editors, for as long as they want to team up. It would also be very helpful to new reviewers. Lara❤Love 22:32, 8 October 2007 (UTC)
I think the idea would only work if the two reviewers were really working together, in real time, simultaneously on the same review. A bit like the XP approach to software development. I don't see much merit in the "I'll check grammar and you check everything else" approach. --Malleus Fatuarum 01:59, 9 October 2007 (UTC)
I don't like the idea of "issues/disagreements between the two reviewers should be handled on their respective talk pages and not the article's talk page." Even such debates should be transparent, and should occur on the article's talk page. See my suggestion above for revamping PR. If we can get PR to be a more widespread and utilized process (from the reviewer end. Plenty of articles get nominated), and we can get discussions to occur on article talk pages, not on some obscure side page or user talk pages, we can get multiple eyes on an article. Otherwise, however, the "team reviewing" idea is a very good idea. --Jayron32|talk|contribs 04:22, 9 October 2007 (UTC)
Regarding the disagreement discussions, I see this as a potential problem with tandem reviewing. There will inevitably be disagreements between reviewers in some cases. It would be unfortunate for one editor to undermine the recommendation of their review partner because they disagree with their interpretation of the criteria, for example, thus sparking an argument which could be an embarrassment to the project. I don't think every aspect has to be transparent; Not all disagreements have to be on display for everyone. It can be talked out on user talk pages, and any changes to the recommendations then made to the article talk page. The review discussion is not the proper place to work out such differences. Lara❤Love 15:54, 15 October 2007 (UTC)

Want to reduce the GA backlog?

Extend the quick fail criteria; for a short time I got involved in doing GA reviews (have since been busy building a new article up, offline), and many times I encountered articles that, on the very first reading, I knew weren't eligible for GA. But, none of them met the quick fail criteria, so I felt obliged to do a full, and time consuming, review, even though I knew from the start that the end result would be a fail. I think that basic prose requirements, as well as MoS strictures, should be brought into the quick fail criteria so that the GA list can be cut down quickly. There will still be borderline cases where on-hold would be applicable, but so many of those nominations should be failed on first sight!

What is frustrating to contributors to wikipedia, who nominate articles for review, whether it be GA, PR or FA is the sheer time it takes to get a response; by being able to quickly cut out the dross, the quality articles will be able to get the attention they deserve, with the result of happy contributors, a better WP, and smarties all round!

Just my tuppence (or 1 cent, to the Americans) worth. Carre 23:26, 9 October 2007 (UTC)

Just a personal view, but I don't actually mind reviewing these sort of articles. Taking the time to point out MoS, style etc deficiencies is beneficial in that it educates editors; the personal touch works better than expecting all editors to read and understand the entire MoS for themselves. The net result is higher-quality submissions next time around, and an overall improvement in article quality. I agree that this may not help with baclogs, but IMO it pays off in the long run. EyeSereneTALK 08:07, 10 October 2007 (UTC)
I am gonna agree with EyeSerene there. I have often left some rather extensive reviews at some really crappy articles. I find the idea of "quick-fail" criteria an invitation to laziness. I have always felt that it is my responsibility as a GA reviewer to provide enough information about a failed article's shortcomings to actually make it a GA if my recommendations are followed. Quick-failing an article does not actually lead to much improvement. --Jayron32|talk|contribs 03:16, 11 October 2007 (UTC)
Going along with what EyeSerene & Jayron32 have said, I don't think expanding the quick fail criteria is the right way at all. I just reviewed Auckland, and considered a quick fail at first (partially because it was simultaneously nominated for FAC & PR & GAN. all at once; and partially because it just fell short of the criteria in many ways). But in the end I did a full review, because I thought the editors were inexperienced and I was able to suggest many ways in which to review it, and lots of resources for them to look at. So hopefully the editors will improve upon this and come back. Dr. Cash 05:22, 11 October 2007 (UTC)
Yep. Agree with Dr Cash and Jayron. If it's gonna fail it's gonna fail - a quick fail would have precluded the helpful comments above. cheers, Casliber (talk · contribs) 06:21, 11 October 2007 (UTC)
So true. We don't want to increase the number of quick-fails. That's very discouraging to article writers. We have the fail option, we need to use it. Articles that clearly fail to meet the criteria can always be brought up to standards in time. If it appears within a few days, then there's the on hold option. Otherwise, it can be failed. But the custodians will, ideally, take those recommendations as tools to build a better article, then renominate it. To expand the quick-fail criteria would be detrimental to the process. It would effectively stop some writers in their tracks because they'd be left with no building materials, so to speak. Lara❤Love 16:19, 15 October 2007 (UTC)

Clarifying the issues in hand - Content vs Style, general reviews vs expert reviews

All too often quality is presented as being a single "dimension" - going from "---- awful" to "perfect". But it's much more complicated than that. I'd like to lay out how I see it - others may see it differently, but I think any delineation should hopefully clarify things a bit. I'm not trying to stimulate a huge discussion (though by all means discuss it if you think it useful!), rather just show the lay of the land beyond the PR and GA/FA aspects discussed above. I just think that before we change the system we need to define the parameters clearly. The parameters mainly fall into two main categories: Content (which requires good knowledge of the topic) and Style (which requires good knowledge of English and WP:MOS). Here are the parameters as I see them:

  1. Content coverage: In the early stage of article development, this is what is needed most of all. As the article matures it moves towards "comprehensive coverage" and this aspect becomes much less important. The best people for judging this are subject-experts.
  2. Content quality: The issues may be covered, but there are sloppy definitions or actual errors. Again, subject-experts are critical here - preferably experts with good access to reliable sources.
  3. Content trustworthiness: The article may be beautifully written and absolutely correct factually - but if there are no clear citations from reliable sources, how can I know that? Many of our early FAs fall into this category. Also see the Quality versions proposal.
  4. NPOV: Related to trustworthiness, but you can still write a biased piece that is well cited. Requires a consensus from several subject-experts, IMHO.
  5. Article scope: Sometimes a topic is better covered fairly broadly, sometimes it's better to break it up into sub-articles, usually with a top-level article using summary style.
  6. Article organisation and flow: The article may be well written and contain all the important content, yet it looks like it was written by a committee (in effect, it was!). It's a side effect of the fact that our articles are written by many people, often in small disjointed pieces. Needs one or two dedicated author(s) - a subject-expert and someone who can write well.
  7. Quality of writing: Issues here include poor grammar, spelling, verbosity, poor use of paragraphs, etc. Good copyeditors needed here.
  8. Level of writing: A popular or general topic like atom will need to be accessible to a popular, general audience. In a more specialized article like Persistent carbene, it would be ridiculous to explain what an electron is. Ideally needs a subject-expert who can write well.
  9. Aesthetics: Sometimes a nice diagram can really help, or a photograph can bring an article to life. Hard to quantify!
  10. Compliance with house style: The article may be perfect in every way, but have a Heading Like This instead of One like this, or lack the   before units, etc. Requires someone with a good knowledge of WP:MOS.

Article length is often informally used as a parameter, but it's largely based on three of the above: Content coverage, Scope and Quality of writing (specifically the conciseness/verbosity part). This is a tricky issue when assessing an article for quality - we may say, "It's long, so it must be pretty comprehensive" but it may fail to cover major aspects and just be very long-winded. If the scope is very narrow it may be hard to make it long enough to get on the radar screen for GAN/FAC - for example, compare Jarvis Island with United States.

Review types

When assessments are performed, the general reviewer can best judge the Style parameters: Quality of writing, Level of writing, Aesthetics, House style and perhaps Article organisation/flow. Therefore GAN and FAC (with mainly general reviewers) focus on these aspects. This works usually because by the time an article reaches this point it usually has most of the content aspects covered. Meanwhile the WikiProjects help to ensure that the Content parameters are covered: Content Coverage and Quality, Trustworthiness, NPOV and perhaps Article Scope and Organisation. That is why Stub/Start/B/A reviews mainly focus on those aspects, and it explains why an article can be A-Class while failing GA (or vice versa). Content issues are most important in the early stages of an article's development, so it is appropriate that much of the assessment occurs at the Stub/Start/B level. Nevertheless, I think the A-Class review system such as is done at WP:MILHIST is extremely valuable as a form of expert peer review, because it ensures that the content aspects are all well covered before the general reviewers at FAC even see it. This is analagous to peer review for scientific publication, before the house style and grammar issues are cleaned up by the copyeditors working for the publisher.

Soon we will have a third main "category" to worry about - the specific article Version - but let's leave that until WP:FLR becomes a reality (next year?)!

Well, I probably missed a few things, but I think it's important to try and lay these things out. It's especially important to realise that Content issues and Style issues are different "beasts" requiring different types of review. Walkerma 05:57, 13 October 2007 (UTC)

Some very good points there. "Quality" is a difficult chap to get a hold of though, very hard to judge or to measure. It's much easier to see the lack of quality in an article than it is to see its presence. Not helped by some of the daft rules of thumb that some reviewers seem to depend on, like correlating length of article with length of lead, or number of citations with number of paragraphs, for instance. --Malleus Fatuarum 09:02, 13 October 2007 (UTC)
This is a fantastic piece of in-depth analysis, Walkerma, of the goals and challenges of review processes. In particular, you raise the distinction between review of content and general review, noting that processes such as GA and FA are not equipped to certify comprehensiveness in many cases, since this requires subject experts. Some time ago, several people raised the question whether this informal separation of roles should be made more explicit. In other words, should the GA/PR/FA system be woven into the Stub-Start-B-A system, or should it be orthogonal (i.e., a different dimension, in your terminology)? Unfortunately, these ideas were raised at a time of heated debate, and did not lead anywhere. There was some fall-out, though. The math project created a B+ quality grading to orthogonalize the GA process: GA-Class is now used to refer to articles whose mathematical content is B+, and whose general quality has met the good article criteria. I wonder if such a separation of roles would be helpful more widely. Geometry guy 18:41, 13 October 2007 (UTC)
I've never understood.. why all the hullaballoo (sp.?)... I put it to you in brief: what's the diff between GA and A rating? Answer: expert (or domain-knowledgeable) review. GA is purely about form; A is purely about content (or should be; but but of couse A-reviews deal with form when the GA process is skipped or went awry); FA is about superior combination of all of the above..
  • I don't entirely agree with that distinction between reviewing for content and reviewing for form. Both GA and FA should be doing both. Remember that this is an encyclopedia for a general readership, not a specialised audience. If a reasonably knowledgeable reviewer can't understand the article then it's probably not suitable for the encyclopedia. --Malleus Fatuarum 12:02, 14 October 2007 (UTC)
I think this is a particular issue with maths/science articles, many of which are not written for even the educated layman. I've seen it explained as "why would a general reader want to know about X anyway; it's a specialised subject" (my paraphrasing), and I understand that one can't explain everything from first principles every time, but I'm not sure I can subscribe to that view. This is not to say that such articles shouldn't need subject experts to review content (any more than eg ancient history articles would), but rather that they would benefit from a 'plain english' copyedit as part of the review process. Back to collaboration again ;) EyeSereneTALK 12:25, 14 October 2007 (UTC)
Understanding an article is often not enough to assess its content for neutrality and breadth or comprehensiveness. Expert reviewers are essential for content review even for articles which are entirely comprehensible to the lay reader. Geometry guy 18:51, 14 October 2007 (UTC)
By the way I'm very very late to this party but wanna refer back to G-guy's comment way up the page: I too strongly prefer more proactive GA rather than dictums from the clouds. Work on those articles. I stand against all the new conventional GA wisdom in this, apparently. [which I confess kinda pisses me off.. no not kinda.. definitely].Cheers! --Ling.Nut 11:38, 14 October 2007 (UTC)
Eh, let's not nitpick. Please. GA review/rating should be mainly about form (and oh yes it touches on content too but not as a main focus!); A review/rating should be mainly about content (and oh yes it touches on form too but not as a main focus!). Nitpicking adds no light to the conversation, and creates many side-tracks.--Ling.Nut 12:52, 14 October 2007 (UTC)
GA and A-class are very different. It cannot currently be expected of either GA or FA to achieve the sort of expert review that results in A-class. The vast majority of GA and FA reviewers do not have the subject knowledge to determine this. The only way to achieve this would be to solicit the involvement of experts for reviews of articles in their field, which would basically be merging A-class into GA and FA. To expect an article to be "dumbed-down" (for lack of a better term) to reach a "general audience", I can't agree with that. There are some subjects which contain a necessary amount of what is considered jargon to a general audience, but is very much necessary for readers interested in such a topic. This is an encyclopedia for everyone; that includes everyone from grade schoolers to academics. Lara❤Love 16:48, 15 October 2007 (UTC)
There is a very big diference between "dumbing-down" and writing intelligibly; nobody has suggested any "dumbing-down", and to characterise a very reasonable concern as if they had is disingenuous. --Malleus Fatuarum 16:56, 15 October 2007 (UTC)

(←) You're quite right that "dumbing-down" is very different from "writing intelligibly". I stated "(for lack of a better term)" because the use of "dumbing-down" was my own. I wasn't quoting anyone, just as your use of "writing intelligibly" is not quoting anyone. Additionally, I realize "dumbing-down" may convey more of a negative tone than would be preferred, which is why I put the disclaimer there. Past that, you stated "Both GA and FA should be [reviewing for content and reviewing for form.] Remember that this is an encyclopedia for a general readership, not a specialised audience." Most reviewers don't have the expert knowledge necessary to adequately review articles for A-class, which is why that rating is only awarded by projects. And, as I previously stated, and as I now see G'guy had already commented on, this isn't an encyclopedia for only a "general readership". Lara❤Love 20:14, 15 October 2007 (UTC)

A very important aside

This is not really relevant to the content review process specifically, but is so important to the entire mission of Wikipedia that I must mention it here to counter the plainly incorrect assertions made above that "Wikipedia is an encyclopedia for a general readership, not a specialised audience", "If a reasonably knowledgeable reviewer can't understand the article then it's probably not suitable for the encyclopedia", and that it is a problem if some articles are "not written for even the educated layman".

Wrong wrong wrong. And I am not expressing a point of view here.

"Imagine a world in which every single person on the planet is given free access to the sum of all human knowledge. That's what we're doing." (Jimmy Wales)

Note that he does not say "all human knowledge which is understandable by the average reader".

The "Five Pillars" make this utterly explicit.

"Wikipedia is an encyclopedia written for the benefit of its readers. It incorporates elements of general encyclopedias, specialized encyclopedias, and almanacs." (Pillar One)

In WP:NOT and WP:N, the meanings of "knowledge" and "encyclopedia" are clarified by excluding certain types of information, such as directories, and non-notable content, but nowhere is "knowledge" limited to that which is understandable by the general reader or educated layman.

I suspect that those who think otherwise are mentally comparing Wikipedia to Encyclopedia Britannica, but Wikipedia is not paper: EB is 20 times smaller than WP, with only about 70000 articles on 1/2 million topics. I find it more helpful to think of WP as a nested family of overlapping encyclopedias, rather than a single monolithic encyclopedia.

Of course, we should strive to make every article as widely accessible as possible, but this is not a criterion for inclusion. Geometry guy 13:35, 14 October 2007 (UTC)

Point taken; Wikipedia is not Wikiversity, although I do think technical/specialised articles should at least include enough non-technical explanation and background so that the layman can understand why they are notable. My argument is not so much about content, but the way in which it is presented. EyeSereneTALK 14:37, 14 October 2007 (UTC)
Quite. Geometry guy confuses "knowledge" with its presentation. And no amount of "Wrong, wrong, wrong" statements such as the rather patronising one above will persuade me otherwise. --Malleus Fatuarum 16:45, 14 October 2007 (UTC)
I agree entirely with EyeSerene, and it was precisely this point I was alluding to in the last sentence of my post. I'm sorry that you found the emphatic tone of my argument patronising. That was not my intention. I spoke stridently because this is one aspect of Wikipedia that I am passionate about. However, I assure you, I am absolutely not confusing knowledge with presentation. Every article should be presented in as widely accessible way as possible, but that does not mean that material which cannot be presented in a way that is accessible to the general readership should be omitted. It would not be omitted in a specialist encyclopedia, so it should not be omitted here. Geometry guy 18:45, 14 October 2007 (UTC)
Yes, I agree. The beauty of Wikipedia is that there is room for an article on persistent carbene as well as one on atom. I also agree that for a specialized article, the introduction at least should be accessible to the lay reader; there's no way you can have a meaningful article that is completely accessible, though. That's why we need both types of reviewers - specialists and generalists - and ideally they should agree that the level of writing is right. Personally, when I write a specialised chem article like gold(III) chloride I try to write an intro at the high school level, then some basic content at the undergrad level, but then add some valuable recent material for the professional chemist. And think about how many non-chemists will go searching for an article on gold(III) chloride or persistent carbene...! Walkerma 16:40, 14 October 2007 (UTC)
I largely agree, except for one thing: WP:LEAD. The lead should summarize the article, which means that for some articles the lead will not be entirely accessible to the lay reader. However, often this can be componsated by having introductory material in other sections. Geometry guy 18:45, 14 October 2007 (UTC)

Straw poll for Current Topic

I'd like to go ahead with a straw poll to pick a topic for us to focus on, per the discussions above. I didn't hear anyone suggest that a focus topic was a bad idea, so let's try it.

My suggested format would be: paste in the list of problems/topics on the project front page, and simply put Support followed by your name underneath the one you think we should focus on next. Sign under more than one if you'd be happy to focus on either one; I suggest we pick only one to focus on at once, though. Discussion (e.g. "This one is a bad choice because we can't fix unless problem X is fixed first") can take place under each one too, with Comment prefixes if necessary.

Here's the list, with my support in place. Mike Christie (talk) 13:02, 14 October 2007 (UTC)

  • How can reviewers be encouraged at all forums? Peer review in particular is perennially understaffed.
    • Support. Mike Christie (talk) 13:02, 14 October 2007 (UTC)
    • Half-support. Not "at all forums"—too much for a single current topic. I support starting with PR for all of the reasons listed above.
    • Support something along the lines of "how can we improve the efficacy of peer review?" Awadewit | talk 16:54, 14 October 2007 (UTC)
    • Support I think this is probably one of the key issues that need to be addressed in wikipedia's overall review system, as a whole, at all levels. Dr. Cash 06:17, 15 October 2007 (UTC)
    • Support This is the weakest area right now, IMHO. Walkerma 15:29, 15 October 2007 (UTC)
    • Support (per building consensus) as an alternative to starting with the labels/quality scale EyeSereneTALK 16:28, 15 October 2007 (UTC)
    • Support Lara❤Love 16:57, 15 October 2007 (UTC)
    • Support Even though some other topics may be more important or urgent, I think this is a good place to start. Geometry guy 18:54, 15 October 2007 (UTC)
  • Is there a way to reduce redundancy between the FA and GA processes?
    • Is there redundancy? If so, is that redundancy to the detriment of the project? [added by Awadewit | talk]
    • I agree with Awadewit. I'm not convinced this is a problem. Lara❤Love 16:57, 15 October 2007 (UTC)
  • How do we recognize the value and quality of good, short articles that are unlikely to grow?
    • Support. Mike Christie (talk) 13:02, 14 October 2007 (UTC)
    • Not yet. This is important but it is best examined in relation to GA, which can wait until after we've built trust this workshop can work. Marskell 14:30, 14 October 2007 (UTC)
    • Comment I'd appreciate if someone could detail how GA doesn't fill this need. Lara❤Love 16:57, 15 October 2007 (UTC)
      Maybe it does, and maybe the answer to the question is simply "Using GA", but part of the point of this workshop is to abandon our preconceptions. Anyway, I agree, this is one for later. Geometry guy 18:54, 15 October 2007 (UTC)
  • How can we encourage the connection of subject-matter experts with articles that need their expertise?
    • Support I actually think this coincides with the discussion on how to encourage reviewer participation. Lara❤Love 16:57, 15 October 2007 (UTC)
    • Support This would be a natural follow-on from the current PR participation topic. Geometry guy 18:54, 15 October 2007 (UTC)
  • How can all the processes scale? Each has its own barriers to getting ten times more productive.
    • Comment. This is one of the most important to me. I think we need a process, and it could be GA, which can handle 10 times as many articles as the current FA process, but I think we need to think about some of the other questions first. Geometry guy 18:54, 15 October 2007 (UTC)
  • Acrimonious arguments at individual processes and in discussions between regulars at the different talk processes drain time and energy; is there a way to eliminate or reduce these source of friction?
    • Support I think this is important to work out before we advertise these discussions project wide. Lara❤Love 16:57, 15 October 2007 (UTC)
      I agree, but I think that discussing this, and advertising project wide, should both happen much later. Geometry guy 18:54, 15 October 2007 (UTC)
  • Can we avoid or mitigate the weaknesses of single-reviewer systems in which one user may mistakenly pass an article that does not meet the relevant standards?
  • Can we avoid or mitigate the weaknesses of multiple-reviewer systems in which some undocumented and unrecorded form of consensus has to be decided on by a single individual?
  • What can be done in all processes to reduce bureaucracy and instruction creep and make things easy for the content-writers wishing to use the processes?
  • Can statistics be improved for any of the processes, in order to help measure their success? No process keeps complete statistics on both articles reviewed and reviewer participation, but there is great variation among existing processes.
  • Are there opportunities for collaboration between the various review processes?
    • Support I believe this also coincides with the discussion on encouraging reviewer participation. Lara❤Love 16:57, 15 October 2007 (UTC)
    • Support This would be a natural follow on from the current PR participation topic. Geometry guy 18:54, 15 October 2007 (UTC)
  • Should there be a unified review hierarchy?
  • What can be done to make it easier for editors to make articles comply with the somewhat technical manual of style?
  • Should labels themselves be examined—stub, start, B, A, Good, Featured?
    • There would be some sense in starting with this. Marskell 14:30, 14 October 2007 (UTC)
    • Support and widen to include nailing down exactly what these labels mean by examining the quality scale definitions as well EyeSereneTALK 15:04, 14 October 2007 (UTC)
      • Comment An issue here is that the quality scale varies by project. Lara❤Love 16:57, 15 October 2007 (UTC)
    • erm, Support, but as is usually the case in wikipedia forums, wouldn't this discussion overlap with the talk page of Wikipedia:Version 1.0 Editorial Team/Assessment? --Ling.Nut 15:13, 14 October 2007 (UTC)
    • Support This would be good to do even simply for mutual education (we have a wide variety of experience and opinion here). Geometry guy 18:54, 15 October 2007 (UTC)
  • Support - agree that PR is weakest link at the moment. In fact, if it were more active I doubt this page would be happening at all. cheers, Casliber (talk · contribs) 20:31, 15 October 2007 (UTC)
  • Can we improve the mechanism for nominating articles for review, either for each process individually or across the project as a whole?

will never happen 'cause of culture clash, but logically PR should be merged into GA

  • ..uh, like the header says. As I said, GA is about form, A is about content, FA is about all of the above. PR overlaps the goals of FA and GA and A all at the same time. Yes it does.
  • Mike's thread suggestion about lack of reveiewers is easily handled by merging PR into GA. But it will never happen. Territoriality. Culture clash. People are too proud of being PR and too leery of all the knocks on GA (which are also caused by pride, mainly.. people are insulted by being rejected for a rating that just means "Acceptable")..
  • 'Nuff said. --Ling.Nut 13:09, 14 October 2007 (UTC)
"People are too proud of being PR..." You think? I've never thought a "we at Peer review" attitude exists. Marskell 14:33, 14 October 2007 (UTC)
If that's wrong, then I am happy... but.. explain the three-part mission of FA, A and GA, then show that PR overlaps, then float a merger into GA and see what happens... --Ling.Nut 14:36, 14 October 2007 (UTC)
I haven't seen much of a "community" among PR reviewers, so I don't think that argument holds much weight,... Dr. Cash 17:30, 14 October 2007 (UTC)
Actually, this is an interesting idea. As Cash noted, there isn't much of a community of PR reviewers. But GA is well established; perhaps by combining the two, you could solve the dearth of actual human reviews at PR and streamline some grading. So you could send it in for a review, and if could either be simply reviewed or reviewed and passed. I think it merits a discussion, at least. David Fuchs (talk) 17:48, 14 October 2007 (UTC)
I think it's a good principle to have the topics we work on be problem statements, rather than solution suggestions -- we should be trying to fix the problem, and there could be multiple possible solutions. What's the best problem statement that this solution is trying to fix? Increase participation at PR and GA? Increase collaboration between projects? Mike Christie (talk) 18:58, 14 October 2007 (UTC)
GAC can get backlogged, but it's fairly streamlined, and since it only requires one reviewer the only time-consuming part is going back and sampling GA-class to maintain the list. PR, however, is always suffering from a dearth of people, more so than even FAC- I've sent five articles to PR, and only two have received human comments- and one of those, I specifically asked an editor his thoughts. Perhaps by combining them, we could get more useful suggestions on improvement for GAC articles, as well as passing them. David Fuchs (talk) 19:40, 14 October 2007 (UTC)
This is assuming that one gets good reviews at GAC. I have received, overall, abysmal reviews there. That was one reason I joined the project - to offer good reviews. I often receive two lines or a little template with icons filled in saying how great the article is. That is not particularly helpful. Someone once actually ran a bot and posted the results as their review. I'm not so sure that combining the two would solve the problem of a lack of peer reviewing. Awadewit | talk 19:58, 14 October 2007 (UTC)
Is this two separate problems being discussed? One is the need for a way to establish whether or not an article is at a given standard. That would be GA, since the goal is to establish if an article is GA or not. GA reviewers do often add very good review comments, but not always, and they're not obliged to by the process. The other problem is the need to get high-quality feedback on an article, regardless of the standard that it may be at. That's what PR is supposed to do. Awadewit, it sounds like you're lamenting the lack of good feedback at PR, and David, you're suggesting that combining the two would give streamline the grading of articles at PR. But are those two goals really combinable? Mike Christie (talk) 22:16, 14 October 2007 (UTC)
I would agree that these are two separate problems, but I am not lamenting the lack of good feedback at PR. In fact, I've almost always received better feedback at PR than at GAC (again, this would only be about 10-14 data points). And in fact, GA reviewers are encouraged to add comments for how to improve the articles. I'm not exactly sure why someone would use GA if they weren't getting a review. Few people seem to use it for its original purpose - articles that are not appropriate for FA. If GA is only a rubber stamp (either pass/fail), that should be made clearer, but I was under the impression that it was not, since there are "holds", etc.
I do think that ranking and peer reviewing should be kept separate, not just because they are separate processes, but because of wiki-culture. I was a part of the WikiProject Biography Assessment Drive this summer and there seems to be a sizable minority (even a majority?) of editors who dislike the entire ranking process. They should not be forced to go through it just to receive a peer review. Awadewit | talk 22:26, 14 October 2007 (UTC)

I agree with Mike about the best way to proceed here. I also agree with Awadawit that some GA reviews can be two-liners, which is unsatisfactory, whether the result is pass or fail — and I do think this is unsatisfactory, and that GA reviewers should provide feedback. Many GA reviewers are extremely good at this and I think the two goals are compatible.

My own view about GA is that the process should be simplified, that the GAN and GA pages should be automatically generated from talk page templates, and that the process should concentrate on providing quality reviews through interaction between article editors and an independent reviewer on the article talk page. I think the GA and PR processes could be made more similar. This could be a prelude to a merger, or not. Geometry guy 22:30, 14 October 2007 (UTC)

  • Knock knock. Hi Awadewit. :-) My point had nothing to do with the quality (or lack thereof) of GA reviews.. I was a voice in the wilderness ranting for a GA training program many moons ago... and if you read my comments in this thread, I never mentioned quality of review.. but rather, I said that the logical mission of PR overlaps those of FA, A and GA. [If you wanna get rid of ratings altogether, then go for it.. but if the ratings stay, then PR logically overlaps].
  • As almost always is the case, I mostly agree with G-guy. I hereby accuse myself of being his bad-hand sock. Would someone please checkuser me? I need a reality check. Cheers! --Ling.Nut 23:52, 14 October 2007 (UTC)
Great minds think alike! (Or should that be "fools seldom differ"?) :) Geometry guy 00:02, 15 October 2007 (UTC)
GA training program? That sounds like a great idea! When was that? Even if GA and PR overlap, I think encouraging people to get multiple reviews for the article they are working on is good. I'm not sure that "streamlining" will improve quality. The more (good) eyes reviewing, the better I have found. Awadewit | talk 00:08, 15 October 2007 (UTC)

(undent) It happened on two separate occasions, months apart. Not sure where those threads are (not important anyhow), but they were further examples of me babbling incoherently into the wind in GA-related discussions. :-) ...But I still think GA and PR should be merged for the sake of unified mission/efficiency... fold PR into GA if you keep the ratings system as a whole; otherwise vice-versa. --Ling.Nut 00:14, 15 October 2007 (UTC)

One of the goals of a good creative writing class, I've noticed, is to teach aspiring writers how to critically read. By this, I mean developing a skill similar to an engineer's ability to look at an engine & understand how it works. I suspect many Wikipedians -- even many who have successfully written FAs -- do not possess this skill in a form that they can easily apply to articles on other topics. I can personally attest that the most useless form of criticism is a comment along the lines of bland, unengaged compliment like "This is okay. It's as good as anything that gets published now days." Even nit-picking over MoS trivia is better than that. -- llywrch 17:41, 15 October 2007 (UTC)
I've whistled in the wind in a similar vein every six months for two years! The strongest argument against the merger, however, was provided by qpb on the village pump: we do need a place for simple reviews that are not tied to pass/fail. Anyway, it's becoming apparent that willy-nilly PR has become our first Current topic. Shall we make it official? Marskell 15:13, 15 October 2007 (UTC)
I think so. Perhaps we can state the Topic as "What can be done to increase and improve reviewer participation at PR?" How does that sound? Mike Christie (talk) 15:16, 15 October 2007 (UTC)

(undent) I'll go along with the crowd w. respect to a starting topic. I will say that I think that finding some way to define FA/A/GA so that there is a very clear separation of responsibilities etc. is clearly more important, IMHO. Ummm.. why? Because I'm tired of yakking about what is GA, what is A, how is A diff from Ga and from FA yadda yadda. Even a semi-arbitrary border line drawn between them (see my repeated comments above about one is form one is content etc.), if respected, would be a Good Thing... But if we wanna talk about PR, then I say go for it. Cheers. --Ling.Nut 15:29, 15 October 2007 (UTC)

I understand you perfectly, Ling—the lack of demarcation has bothered me to no end. Procedurally, we do need to go one-by-one if we're to accomplish anything and since people are already talking about PR, let's do it. "What can be done to increase and improve reviewer participation at PR?" seems fine. Marskell 15:35, 15 October 2007 (UTC)
My preference was to start there too, as it seems more fundamental to the review process - roles logically come later. However, it also makes sense to go where the general interest obviously lies - !vote added above ;) EyeSereneTALK 16:36, 15 October 2007 (UTC)
I agree this is fundamental, but that does not necessarily mean we should start with it. We can always revisit questions later on, and I think we are starting in a good place for now. Geometry guy 18:59, 15 October 2007 (UTC)

I don't see the point in merging PR and GA, completely. Some editors prefer to skip GA and go directly for FA, getting a PR first. Additionally, not all GA reviewers know the FA criteria, and I'm not sure they should be required to learn it. However, I can see an advantage in connecting the processes to a point. Once an article has passed GA, if it shows potential for reaching FA, it would be helpful if the reviewer then noted, "I recommend nominating this article for FA after the following changes have been made..." Some reviewers already do this, but if this became a standard, PR would get less traffic and, hopefully, would be able to provide more reviews for fewer requests. It would also potentially increase the number of editors going for FA. Lara❤Love 17:25, 15 October 2007 (UTC)

Agree on the last point. When I review at GA I often highlight a few things so it can be pointed in the right direction of FA. Maybe worth mentioning on GA as a good point to have in a 'how ot review' guide? Ultimately this is a volunteer project and time is limited. With this and a few more PR reviewers I see this as the only way forward that doesnt involve a mass of bureaucracy and discussion....tons more than all this so far..cheers, Casliber (talk · contribs) 20:34, 15 October 2007 (UTC)

I'm going to speak against the point in this thread for a very specific reason: sometimes it is important for another Wikipedian to throw out an idea about an article, without worry that either party -- the creators or the reviewers -- are responsible for the idea.

My point is simple. On one hand, there's the frustration that a person submitting an article to FAC feels when some (noun deleted since we're all practicing WP:AGF here) raises an objection over a trivial item from the Manual of Style. If an article is otherwise praiseworthy, but said person made some minor mistake (e.g., used the wrong era system, or regular dashes instead of em dashes), there is the implication that unless this minor flaw is fixed the article will not be recognized as an Featured Article. And sometimes the reasons for the flaw are understandible. As an example, I've been working on Tekle Giyorgis I of Ethiopia‎, & happened to notice someone requested an image of the subject be added to the article; based on my knowledge of Ethiopian history I doubt that such a picture exists, although I would add an image if I found one. Sometimes people ask for things that simply can not be granted.

On the other hand, sometimes the only way an editor can know that there is a problem with an article is if a reviewer is able to leave a comment, without worrying that she/he will be penalized for asking a stupid question. I know that I've made some comments above that sound as if I want to bite my critics, but I'd rather hear about flaws in what I write -- real or otherwise -- so that I have the opportunity to fix them. Sometimes even criticims that is wrong can serve as a pointer to a problem that an informed editor will detect & fix. -- llywrch 02:46, 16 October 2007 (UTC)

Criteria

I'd like to suggest how we move on from the brainstorming list to winnowing out some ideas and fleshing out others. Here are some criteria I'd use to evaluate the ideas above:

  • How big is the idea? If it's a huge change, it is going to be very difficult to tackle as our first proposal. If it's small and perhaps could be viewed as purely cosmetic, it might not test our ability to work together very well. I suggest we pick something big enough for us to feel it could make a real difference, but not big enough to cause resistance just because of the radical nature of the idea. For me, this would eliminate a couple of ideas such as "Scrap PR altogether"; it probably also eliminates "rename PR", which might work in conjunction with something else but shouldn't be our sole proposal.
  • Does the idea clearly focus on the goal, which is to improve and increase PR reviewer participation?
  • Is the idea positive and creative, or could it be seen as negative? We want the first idea we come up with to look and feel positive. I'd suggest this eliminates "banning bots at peer review".

There may be other criteria I haven't thought of; please add to this list. Mike Christie (talk) 18:03, 16 October 2007 (UTC)

I agree, and the first three criteria look good. I also wonder if we need the idea to have some sort of self-contained nature, i.e., that it should not be too dependent on all of the other things that are up in the air. Geometry guy 19:13, 16 October 2007 (UTC)

Something positive and self-contained that might help with encouraging more reviewers (and editors) to engage with the various processes is an even-handed article badging system. Why not have have markers at the top right for every article? What about starting there? Reviewers often put in considerable effort, yet aren't even rewarded by seeing a little graphic on the article for their efforts. If something that straight-forward can't be addressed then I see little hope for this initiative being able to achieve anything. I understand that PR presents a problem in this respect, having no publicly defined criteria and therefore being unable to be ranked against the other review processes. But that circle has got to be broken somehow, as a first step to getting anywhere IMO. --Malleus Fatuarum 01:37, 17 October 2007 (UTC)

Although I don't know the details, I know there are people (such as Raul654, the FA director) who strongly oppose even the FA star, and the idea of adding other metadata to mainspace is apparently one that's been debated repeatedly and never wins sufficient support.
However, I don't see that this would address the topic under discussion. It's not article writers that the current topic is trying to address; it's reviewers, and they are not rewarded by those stars. Wouldn't your argument, applied to encourage reviewers, imply user-page badges for the reviewers, not mainspace graphics for the articles? I know you say the reviewers would be motivated by mainspace graphics but it's peer review we're focusing on. Mike Christie (talk) 02:03, 17 October 2007 (UTC)
I would suggest that anyone would be likely to want to see that their efforts made a difference, both reviewers and editors. Right now there is hardly any, and none if a reader chooses not to click on the article's talk page. I'm suggesting that it's peer review that's the impediment to that kind of encouragement, because it doesn't fit into any kind of hierarchy.
I'm also aware that there are deeply entrenched positions here, and old wars being re-played on a new battle-ground. I'm new to wikipedia, and I haven't seen it all before, so perhaps some of my suggestions seem naive. Anyway, I've had my say, and I've had enough of being shouted down. So I'll just wait now to see if you can make a difference. I hope that you can. --Malleus Fatuarum 02:36, 17 October 2007 (UTC)
I didn't mean to imply your position was naive; I just wasn't sure if you were aware of the history. In fact I don't know all the history myself; I'm not a veteran of those debates -- I've just seen references. And of course if we can make a good enough argument we might overturn an entrenched position. I also didn't intend to shout you down; I really would like this workshop to be a place where we can reach consensus by merging ideas, not by winning arguments.
If you feel that article-space flags would encourage more peer reviewers then I suggest we add that to the list of suggested ideas in the brainstorming section. I'm not convinced yet, but it's certainly worth discussing. I'll add it myself if you don't object. Mike Christie (talk) 02:45, 17 October 2007 (UTC)
You've just made me feel once again that I'm trying to talk into the face of a gale force wind. My point was to encourage reviewers in general not PR reviewers specifically. --Malleus Fatuarum 03:08, 17 October 2007 (UTC)
OK, yes, you did make that clear in your first post. I'll leave it up to you whether to add some form of your comment to the list, and I'm sorry I mis-stated your comments just now. Mike Christie (talk) 03:18, 17 October 2007 (UTC)
I won't be adding anything to the list. It's quite obvious what needs to be done with the current review processes, and equally obvious that there's no will to do it. PR is a parallel process to all of the others, that can be asked for at any time, but without being able to award any kind of assesment. The other processes are hierarchical. --Malleus Fatuarum 03:53, 17 October 2007 (UTC)

(undent) I am sympathetic to the suggestion of some sort of symbol or meta-image on the article to show that it has been reviewed.... but, and I say this regretfully, I really feel the idea will become a non-starter if it hides a wider community debate. I kinda waded ankle-deep in the debate about the Green Dot for GA articles, and the conclusion I came away with is this: we have the bronze star for FA, and that is all the meta-images we we will ever have in mainspace, forever and ever amen. [Some people even oppose that star, as mentioned ealier.] The idea just.. won't.. pass through any wider debate. I know it has a logic and an appeal, but it cannot overcome the resistance. --Ling.Nut 04:20, 17 October 2007 (UTC)

I also think mainspace stuff is a non-starter, but Malleus Fatuarum does raise one issue here, and stresses it below: PR should not be seen as part of the assessment hierarchy, but as something apart from it that can be used at any stage of article development, not just to feed FAC. I think this idea should not be forgotten: I think it would be good to clarify PR's role in this way. Geometry guy 21:43, 17 October 2007 (UTC)

Tweak suggested

WikiProject Biography peer reviews

Would anyone object to me editing the bottom of the Peer review instructions as below? It would enable me to stop transcluding the WikiProject Biographies to this page, as they would show up in the announcement list instead. DrKiernan 08:12, 17 October 2007 (UTC)

Related pages:

Topic-specific peer reviews (full list):

Biography peer reviews: Romano Volta • Suzanne Carrell • Mullá Husayn • John Gilchrist (linguist) • Thomas Brattle •


If I understand the change, you're removing the Biography peer reviews from the list of specialized peer reviews and transcluding them directly into the list of reviews; is that correct? If so, the only comment I'd have is that some of the ideas being discussed here might conflict with this approach. But the real issue is that I would think this is a question for PR, not for this group. Mike Christie (talk) 11:55, 17 October 2007 (UTC)
I'm suggesting removing the Wikipedia:WikiProject Biography/Peer reviews currently transcluded to the Wikipedia:Peer review page (e.g. [1]) and instead transcluding this list: Wikipedia:WikiProject Biography/Peer review/Announce. DrKiernan 12:50, 17 October 2007 (UTC)

help!

um, I just finished !voting... but I'm not sure if my non-votes were in the right place.. this page is so out of hand! Why did we split the P1 list etc into Dueling Lists, with one called "nitty gritty details"? err... uhhh.... mmmmm --Ling.Nut 17:33, 19 October 2007 (UTC)

I don't have a map either, but I don't think we're in Kansas any more... EyeSereneTALK 17:50, 19 October 2007 (UTC)
The second list is a follow-up of the first one. We all seem to be agreed in principle, so we need to start working on the details: that was the idea of the second list. Marskell has proposed (and I agree with him) that we consider them one by one, and so started a thread on the first one just above this. Geometry guy 18:04, 19 October 2007 (UTC)
Supp... er... oh! Right! That does help a lot, actually - thanks Gguy ;) EyeSereneTALK 18:13, 19 October 2007 (UTC)

(un..you know) crap, so i hafta go refactor my comments grumble grumble --Ling.Nut 18:20, 19 October 2007 (UTC)

How we do the next bit

I hate to sound like a process geek, but I think we're getting a bit bogged down and the way out is to figure out what the steps are, rather than trying to take the steps.

Here's a way we could go. Please tweak this process if you don't like it.

  • We go through some brainstorming and winnowing process on the Current Topic and come up with some ideas on which there is agreement. (DONE, for this iteration).
  • On the project front page, we create a section called something like "Current Topic proposal". It contains someone's idea of what the current consensus consists of. Anyone can edit that section; talk page is used for discussion. We try to reach consensus on it. level of detail needed is: enough to reach consensus. Any outstanding questions (e.g. technical questions about whether to use a bot or not, or whether to use a parameterized template or not) can be listed on the front page too. We can agree to eliminate each outstanding question, or leave them there if they're not critical to the discussion
  • The next step is to decide A) we have a proposal complete enough to take somewhere (PR, VP, wherever); or we don't, and we need to add more to make it complete. That's a talk page discussion.
  • If we need more, we start again with current topic and keep going. If not, we take it to the target page as appropriate.

Separate idea, not linked to the above: I suggest that this page needs a capital F facilitator. This should be someone who the group trusts not to grind an axe, and who is focused on keeping us moving through the steps. They should probably not take a position if they can avoid it. They should identify consensus and declare it, and keep us moving; they should start each process step. Not everyone is suitable for this role. If we like this idea we can elect someone; it can rotate or change periodically if we like.

Any comments? If we like the process, next step is to make a "current topic proposal" section; if we like having a facilitator, if enough people say so we can try that.

-- Mike Christie (talk) 23:39, 20 October 2007 (UTC)

I like both your proposal and the idea of having a facilitator. --Malleus Fatuarum 23:44, 20 October 2007 (UTC)
I agree. These both seem like good ideas. Dr. Cash 03:54, 21 October 2007 (UTC)
Yup, good proposals both. EyeSereneTALK 08:31, 21 October 2007 (UTC)
I think both are good as well. I nominate Mike. He seems to have been performing this function by default, anyway. Awadewit | talk 08:43, 21 October 2007 (UTC)
Sounds good. Also Mike for the job sounds good. --Ling.Nut
For the second part, yes, Mike for Facilitator! For the first part, I agree with all apart from the idea that as soon as we have a proposal on a particular question, we take it somewhere. We have a long list of interrelated questions about content review, and we are going through them one by one to come up with ideas to address them. The solutions to one question may affect the solutions to another. I think we need to have 3 or 4 of these issues under our belt before we talk about rolling out proposals to other pages, and then we need to think very carefully about how we do it. Saying "there is a problem with your process and we've decided that this is the way to solve it" will almost certainly not work. (That is not to say that individuals here cannot open discussions elsewhere based on individual ideas generated here: indeed that may help to "prime" discussions elsewhere.) Also, we have to guard against an influx of editors to this workshop. We have a great group here who are working together brilliantly in my opinion. Additional contributors are of course welcome, but this page is already getting a lot of traffic, and we need to maintain our focus. Geometry guy 10:55, 21 October 2007 (UTC)
I can take on the facilitator role with a couple of caveats about availability: I will be traveling for a week or two in early November and though I think I should have reasonable net access every day or two, I may go through three or four days at a time without access. I don't think that's a big deal, but I wanted to mention it; my job will cause that sort of access interruption periodically. However, if I do a good job as facilitator things should be moving smoothly with or without me.
I will put something on the front page later today to start capturing the consensus we have so far, and will also try to define the facilitator role. Mike Christie (talk) 12:57, 21 October 2007 (UTC)

bogged down in PR.....

Looks like forward progress is taking place, thanks to many contributors... but hey.. let's not forget there are other major areas of discussion that we have left largely untouched (GA being the primary example). Let's not get too bogged down in PR. I know no one will agree, since this is wikipedia, but I favor setting a time-limit on this PR discussion. One more week would be the most unimaginably vast expanse of time I could possibly bring myself to suggest. Three more days would be better. GA needs a training program, needs... and... and... the task is never completed; only stopped at some reasonable point --Ling.Nut 14:20, 21 October 2007 (UTC)

Next: proposal consensus and implementation details

Marskell has put what seems to me to be an accurate summary of the current consensus on the front page at this section.

I believe we have consensus that a barnstar, at least, is worth doing, and that that can move up into the agreed proposals. The idea of a list of active contributors does not have consensus yet. I will add the barnstar idea to the consensus list unless someone else does (or someone objects).

We need to separate implementation details from agreed proposals. The discussion of templates is important, but may not be of interest to everyone in this group. I don't think it should get in the way of ongoing proposal discussion, but I don't want to defer it since I think digging through the nitty-gritty details can expose some additional questions and inconsistencies that need group discussion.

How about asking two or three of us to work on those details on a sub-page? I think I would propose Geometry Guy and Marskell, and one more person if there's a volunteer. We'd ask for two things from this subpage:

  1. How will it work? Create any templates needed. Write instructions as they would be read by the proposed users, including reviewers and reviewees. Describe the function of any bots needed. Cover archiving, ArticleHistory, anything else needed.
  2. How would it be implemented? Say where the proposal would be made (VP, PR talk, etc.) and write a draft of the announcement paragraph. If the group doesn't feel it should be announced yet, they don't do this step.

This would allow the workshop to go ahead with trying to reach consensus on the remaining elements of the current topic, which I think we could do quite quickly. Mike Christie (talk) 15:30, 21 October 2007 (UTC)

I think there may be consensus for a list: it is just that there is not a consensus for listing reviewers by their interests, or in categories. As a compromise, reviewers could be listed with their interests. As for awards, I agree there is consensus that we should have them, but not how they would be awarded. I've attempted to update the front page to reflect this, but of course it can be changed if my interpretation is wrong. Geometry guy 16:22, 21 October 2007 (UTC)
If it's not listed in the category I think it of little use; we can't expect nominators to browse through a list of dozens or hundreds to find the one reviewer interested in Spoo. If page layout becomes a problem we can use hide/show. Marskell 16:34, 21 October 2007 (UTC)

Geometry Guy, Marskell, since you're both online at the moment, how do you feel about the suggestion that the two of you take the implementation details to a subpage? The two of you have the technical knowledge and, apparently, the interest in the details. Mike Christie (talk) 16:41, 21 October 2007 (UTC)

I'm not completely convinced of the need for a subpage: if people aren't interested in the details, they don't need to contribute to that section. But if you think that a subpage is the best way to go, I am happy to be so-facilitated :-) Anyway, yes, I think I can provide the technical know-how about bots and templates, but we really need some input from a frequent peer reviewer about what is likely to work well in practice. Geometry guy 17:22, 21 October 2007 (UTC)
Mike, do you mean take all of the PR details to a subpage or just the issue of the template for the nominator? As I see it, G'guy and I are sort of having a separate conversation a few threads up so we'd just be doing out of view what we're doing in view now. And there's been progress, as we've decided what the options are; I was going to introduce it as a discussion as had been done successfully before. But by all means move ahead to the next talking point while we carry on. The question of how to list the reviewers seems the logical next step. Marskell 18:53, 21 October 2007 (UTC)
What I'd meant to suggest was that just the technical details could be covered on a subpage; the only reason was that I thought that topic might not be of interest to everyone in this group. If it's not distracting to others to have that conversation going on here, then of course there's no need for a subpage. (As for getting a frequent PR reviewer, who in the group has a lot of experience of PR? I know Awadewit does; is there anyone else who's familiar with it?)
I think the level of agreement we need with this workshop is along the lines of "To be listed as a peer reviewer on the PR page you place template X on your talk page. You can parameterize it by category and add a comment if you wish to be listed under a category." (This is just an example.) That doesn't define how the template will work, or if a bot will be involved; it just says what a reviewer will do. Similarly, for the PR page, a sample layout might be a good idea, showing what the resulting page will look like. I'll see if I can make the front page a little more specific about what is still under discussion and what can be deferred to implementation, and then restart a discussion thread below. Mike Christie (talk) 20:04, 21 October 2007 (UTC)
To preempt you slightly, I've listed the possibilities that G'guy and I had arrived at re nomination procedure. I agree about a sample layout; I think we need to get general agreement about the initial five proposals and they we can make a mock-up. Marskell 20:22, 21 October 2007 (UTC)
Yes, that's a good pre-empt. I've listed what I think are the open questions on the front page; I think the top three or four are probably pretty quick to knock off. I put what I thought would be the hardest ones at the end. After we get consensus on your section below, I suggest we work down the list on the front page.
I'll refrain from putting support or oppose on anything in the spirit of neutral facilitation. Mike Christie (talk) 20:25, 21 October 2007 (UTC)

Some existing problems

Dr.Cash put together a list of problems which are now on the brainstorming page. That list is organized around the existing processes such as GA and FA, and lists perceived issues with those processes. I think it might be useful to identify what the content review methods as a whole are not good at. My top two problems would be:

  • Encourage reviewers. I don't think any of the existing processes are good at this. GA has handed out barnstars for people who've helped clear a backlog; I don't think there's been anything similar at FA or PR. However, as far as I know none of the processes, including GA, have a way to reward reviewers who simply show up and review a lot of articles. I don't know if any of the WikiProjects have any internal recognition for this sort of work. By comparison, the list of FA writers rewards competitiveness in writing FAs, and both FA and GA writers often list their completed FAs and GAs on their user pages.
  • Connect experts to articles needing content review. Only the WikiProjects make an attempt at organizing this. Experts do get involved in particular articles, of course, but at PR, FA and GA you are quite likely to find your reviewer(s) quite ignorant of the subject matter of the article they're reviewing. This makes them a perfectly good sample reader, but it doesn't let them vet the content for accuracy and thoroughness. The active WikiProjects such as military history do review work for A and B class assessments, but I don't know if they send reviewers to FAC and GAC to find and review specialist articles.

Finding ways to address either of these issues would be very beneficial to all the content review processes. Does anyone else agree these are the biggest issues we have in content review? Mike Christie (talk) 09:58, 9 October 2007 (UTC)

For my part I agree that connecting experts to appropriate articles is a major issue. This would possibly, as you imply, involve bringing the individual WikiProjects on board; perhaps they could be approached to supply a list of experts willing to undertake on-request content reviews. Problems: there will always be some areas that don't have suitable reviewers; 'expert' status on WP is often a matter of opinion and WP:AGF; experts will not always agree with each other; and this will almost certainly introduce further bottlenecks into the whole process.
Mostly for the last reason, I can't help wondering if 'expert review' might not be better off as another stage in the review process (perhaps even a post-FA stage). This may at first sight run counter to the principle of this content review, but (at 2M+ articles and counting) for purely pragmatic reasons we'll achieve nothing unless we fit the workload to the available people rather than the other way around. EyeSereneTALK 07:55, 10 October 2007 (UTC)
Regarding the first point (reviewer encouragement), as well as the rewards during backlog drives, GAN also has a regular "GAN reviewer of the week" which certainly generates some interest. (However, it probably needs to add to this a further reward for "GAN reviewer of the week other than Dr.Cash"!) Geometry guy 14:43, 10 October 2007 (UTC)
The first is certainly true—not just encouraging reviewers but finding reviewers. Despite accusations of dark motives, a principal reason I have harped on GA being separate is that a divided cohort of reviewers seems needless. Expert involvement is a different beast and, if we do approach it, I think it should be kept simple for now; type the name of your main source(s) into google and find a university e-mail. A full bureauacracy doesn't seem necessary.
On a different note, it seems from the talk page that there's general agreement that PR needs a rethink. Should we more formally focus on it? Marskell 19:15, 10 October 2007 (UTC)
I'll start a separate topic below to give my answer to your second question since I think it relates to how this group feels it can best achieve good results.
For the question of finding reviewers, I'd be willing to agree this is the single biggest issue we have; recruiting and keeping reviewers. There aren't enough reviewers at any of the processes. My own suggestion for addressing this would be to try kudos, which is about the only thing we have to offer, after all. How about seeing if Rick Bot or GimmeBot or StatBot could come up with a reviewer activity list, analogous to the FA nominator list? Maybe have a new page each month, one for GA and one for FA. Everyone who assesses an article for GA (pass or fail) gets a point; everyone who contributes to an FA nomination gets a point. We could create userpage badges for the top 20 reviewers each month: "I was the 3rd most active GA reviewer in November 2007" or something like that. Annual badges, too. Would that be an incentive? If not, what would? Mike Christie (talk) 03:10, 11 October 2007 (UTC)
Hey, it's not my fault that I sweeped the GAN reviewer of the week for the past month (but thanks to Epbr123 for the award, anyway!). But on a more serious point, I think Mike has some great ideas here; although I would be cautious about automating the actual process of giving awards; it could be abused -- for example, if 1 pt is given for anyone that makes a comment on an FAC, what's to stop someone from posting 100 identical, "Support The article is great!", crappy and useless reviews? But automated scripts would be useful to help track some of this data; an actual human should review it and make the award, though.
Another good award might be to have a "new FA/GA reviewer of the month", specifically given to a new reviewer that has shown not only a good quantity of reviews, but also a good understanding of the criteria in their reviews, and good quality. Dr. Cash 05:32, 11 October 2007 (UTC)
While I think the idea itself is good, the problem comes in with implementing it. I agree that it could be abused if it were a bot tabulating all of the points and someone handing the awards out from that. Dr. Cash is correct, I believe, that a human would need to review everything and make their determination on who deserves what. The problem with that is that this person would be someone qualified to do reviews themselves but would be instead doing this. It would also be a huge task. They would basically be reviewing, even if only scanning, every review for more than 20 editors. Having done something similar to this, I can tell you, it's an extremely tedious task. I doubt anyone would want to do this month after month. After doing a few hundred quality reviews after the mid-summer backlog elimination drive, I can tell you, I never want to do it again. Lara❤Love 16:14, 15 October 2007 (UTC)

Connect experts to articles needing content review. Only the WikiProjects make an attempt at organizing this. Recently, a medical article passed FAC with only one Support, although most medical editors who reviewed the article did not Support. Experts were connected to the article and it was reviewed by content experts. SandyGeorgia (Talk) 01:07, 25 October 2007 (UTC)

Current Topic set to "What can be done to increase and improve reviewer participation at PR?"

I've added a "current topic is" sentence to the project front page lead; it looks like the consensus is pretty clear, though not every active commenter had !voted.

I suggest the next step would be for someone to comb through the threads above and try to extract some statements that summarize some suggestions that have been made -- giving out badges to top reviewers, merging PR and GA, changing the way PR works to make it a list of available reviewers, and so forth. I think it would be good to have a brainstorming step, where we come up with a dozen or so ideas, for a day or so, and then focus on picking the best two or three ideas and really trying to make a proposal out of them. I will try to do this combing through and summarizing step if nobody else has a go at it, but it won't be for a few hours -- I'm just posting at work on lunch break and I won't have time to do a thorough job till this evening, or possibly tomorrow. Mike Christie (talk) 18:12, 15 October 2007 (UTC)

Brainstorming list

Please add ideas (new, or extracted from the conversations above) to this list.

Kudos and encouragement

  • Create a peer review barnstar
    (Like that one - Andy from Little Britain)
  • Create a "top peer reviewers" list
  • How would peer reviewers be evaluated? I worry about this one. Quantity of reviews isn't always the best way and quality is always very subjective....
  • Create a "how to" guide for peer reviewing, with links to good peer reviews of different sorts
  • Ask someone with cachet, such as Jimbo Wales to make a soapbox statement supporting PR

Eliminate PR as a separate entity

  • Merge PR with GA, to bring both groups of reviewers together
  • Scrap PR completely
  • Devolve PR to the WikiProjects

Wording changes, clarifications and reorganizations of existing structure

  • Rename PR to avoid the misleading idea that it requires the reviewers to have some kind of academic expertise in the subject being reviewed.
    (Tricky as Peer doesn't necessarily mean academic...)
    – then someone needs to define what "peer" does mean.
    • "Peer" means "person of equal or similar status" as in "peer group pressure" and "tried by a jury of ones peers".
  • Clarify what is the peer group from which peer reviewers are drawn. It could be all Wikipedians, or it could be only Wikipedians with some degree of expertise in the content of the article under review.
  • Group current peer review requests by subject.
    (Like that one - Andy from Little Britain)
  • Change PR so it simply lists articles for review, but the reviews happen on the article talk pages
  • Use PR as the place to go if an article fails GA but the nominator thinks it should have passed.
  • PR should "recommend FAC" if an article is close to FA.
    • Implicitly, editors use it this way already. Marskell 19:00, 16 October 2007 (UTC)
  • Refocus PR toward user talk, with lists of people to ping in particular subject areas
  • Multi-tier peer review, with defined levels of review (e.g. A to E, with E being simple style checks and A being detailed review)
  • Peer reviewers could list areas they are interested in (as at GAC) and people looking for peer reviews could solicit them, if need be

Augment or modify PR's tools or features

  • Ban all bots at PR
  • Create a script that could be run at PR that would highlight MOS errors interactively
    On this topic, take a look at this diff showing Brighterorange's script at work on a current FA candidate. This is quite impressive. It looks like it would take a lot of tedium out of the necessary business of having a house style. More capabilities along these lines would be great. Mike Christie (talk) 15:28, 16 October 2007 (UTC)

Discussion

I like not signing these; it avoids personal association with the ideas. (The above two were not [all] added by me.) Anybody object to me turning SineBot off for this page? Mike Christie (talk) 18:34, 15 October 2007 (UTC)

Good idea. We are all pretty reliable signers I reckon. Geometry guy 19:02, 15 October 2007 (UTC)
Done; if anyone disagrees, please revert. Mike Christie (talk) 19:09, 15 October 2007 (UTC)
While having a general add-to list is fine, we need to separate changes in degree versus changes in kind. A PR barnstar or a "how-to" are fair but not fundamental (changes in degree). I'm in the camp that thinks PR should be fundamentally reworked (changes in kind). Perhaps, as the next step after deciding on PR for our first topic, we should decide just how much change we want.
So, rather than another boiler plate, a question: do we agree that PR as a random "drop off spot" does not work? Marskell 22:01, 15 October 2007 (UTC)
Yes. The brainstorming list should help address this. –Outriggr § 22:12, 15 October 2007 (UTC)
Yes. PR isn't what it claims to be on the label. --Malleus Fatuarum 23:07, 15 October 2007 (UTC)
Sometimes it is good to have both small and large changes. Small changes can more easily be implemented and then a sense of accomplishment can be felt by all which will fuel the energy for larger changes, which take more time. Awadewit | talk 23:03, 15 October 2007 (UTC)
I don't agree that PR does not intrinsically work, just that currently there are issues making it thus. Initially I thought that maybe it was just no-one was reviewing - then I thoguht that maybe the size of WP now and the anonymity means that there are considerable barriers to a 'friendly neighbourhood PR' covering all WP. (bit like how you talk to classmates in a 30 kid classroom but not a 200 student lecture hall..)cheers, Casliber (talk · contribs) 23:52, 15 October 2007 (UTC)
The above are some good points. Another couple of things to consider:
  • How should the WikiProjects be involved in the process? If you're looking for peer review on an article on Homer Simpson, someone from the Simpsons WikiProject would care much more about working on it than I would. At the same time, the WikiProject may have already been asked or even helped, and the person is looking for help on things like formatting. Any new process needs to be able to handle these aspects.
  • Are there areas where there are successful peer-review-type-processes going on? Is there anything working well that we can try to emulate?
Just my 2c. BTW, I'm off to the French Wikipedia Colloque soon, so I my contributions here may get sporadic. Walkerma 02:51, 16 October 2007 (UTC)

(undent) [question] Hey what are the sweeping changes that Marskell wants? :-) [trivial point: can safely be skipped]: Ummm, rename Peer review to Community Review, as per the concerns about Peer implying expert? --Ling.Nut 04:28, 16 October 2007 (UTC)

  • Would it be useful to separate "internal" and "external" peer reviews in some way? Peer reviews done by people familiar with the subject, who focus on assessing an article's accuracy and comprehensiveness, and peer reviews done by those unfamiliar with the subject, who focus on assessing the readability and accessibility of of the article? Or would that just be confusing and unhelpful? Awadewit | talk 05:16, 16 October 2007 (UTC)
well the GAMedal looks nice (who made that?) and seems to have helped GA somewhat, so a PR barnstar would have to look really nice in order to be effective. But, ahem, where are those stellar beings we refer to as Jimbo and ArbCom etc.? I mean, the only incentive we have is barnstars. Is that enough? Can we think of other incentives? What about moral suasion? Why not some good old-fashioned soapboxing? "Jimbo says, 'It takes an entire village to review an article'" etc etc. Yes I know it's another dumb idea that will never happen. I'm full of them. I majored in dumb ideas. :-) --Ling.Nut 07:09, 16 October 2007 (UTC)
You mean the "for the betterment of humanity" argument doesn't work? I'm shocked. Shocked. :) (How about an award revolving around the Encyclopedie? I can't believe its article is so bad!) Awadewit | talk 07:36, 16 October 2007 (UTC)
I like the 'Community Review' title as being less potentially intimidating than 'Peer Review'. However, I think the biggest barrier to increasing participation is in the first paragraph of the PR page: "[PR] is intended for high-quality articles that have already undergone extensive work, often as a way of preparing a featured article candidate". This alone limits the pool of suitable reviewers.
I suppose this also goes towards the What can be done to make it easier for editors to make articles comply with the somewhat technical manual of style? discussion point, but might PR not be more useful as a wider process that welcomes any article, regardless of its condition? As they (sort of) already are, editors nominating articles can be encouraged to pick another to review themselves, building in scalability in much the same way as GA, and the process becomes more generally useful to all editors. No rankings = no pressure on new reviewers nervous about making mistakes, and we end up with somewhere where anyone can take any article with the question "How can I make this better?". EyeSereneTALK 10:27, 16 October 2007 (UTC)
I thought PR did welcome any article, sort of by default. I'm not sure that many editors read those directions. How many of the 100 or so articles on PR currently are "high-quality"? I support changing the language, but I don't think it will help much because I think people are already doing what we are suggesting. I've seen pleas for "help me get this from start to GA", etc. Awadewit | talk 10:54, 16 October 2007 (UTC)
I'll admit I'm not hugely familiar with PR, but my experience of articles that have used it has been as almost exclusively a post-GA/pre-FA process. It may help to explicitly advertise it as a general improvement and editor education process. Of course, there are editors that nominate to PR, GA, A and FA all at the same time, presumably in the hope that one of the four will get round to reviewing their work in a timely manner. I realise it's not particularly relevant to this discussion, but maybe we should think about having a single clearing-house location for all nominations where the nominator posts the article link and states which type of review they are after... actually, I think I'll go and add this to the discussion list ;) EyeSereneTALK 14:02, 16 October 2007 (UTC)
I think EyeSerene raised a good point up earlier in the discussion. I've been around a while, yet I feel hopelessly ill-equipped to bring an article up to FA standard all by myself (even GA can be a challenge). It's a bit like trying to get 100% on an exam. This makes me feel apprehensive - I feel awkward playing a god-like role for this mortal who wants me to "fix" their article so it meets all the criteria. On the other hand, when a friend asks me to give a few comments and a copyedit I'm happy to get stuck in and help! Now I realise this is all very silly of me, but this topic is all about perception. I think we need to make the system "feel" that we as a community will chip away at the problems until the article is pretty much right.
Regarding Awadewit's point on separating "internal" peer review from "external" review, I think the internal ones are best done through WikiProjects - and this group might collaborate with WP:COUNCIL in promoting these. These can have the same perception problem as I mentioned above, though - I don't want to set myself up as a know-it-all. In chemistry, we've had little success with the formal peer review process, yet informal postings on the WP:CHEMISTRY talk page like "Hey, I just expanded the XXXX article, can you take a peek" usually elicit a lot of valuable improvement. So - how to make the formal process work as effectively as the informal? Walkerma 16:09, 16 October 2007 (UTC)
I agree with you. It's much how I feel. Perhaps making the less formal and more "hands on", as in Ling.Nut's proposal below is the answer. --ROGER DAVIES TALK 16:26, 16 October 2007 (UTC)
Of course, there are editors that nominate to PR, GA, A and FA all at the same time, presumably in the hope that one of the four will get round to reviewing their work in a timely manner. Per the instructions at both WP:PR and WP:FAC, they shouldn't be doing that. SandyGeorgia (Talk) 01:31, 25 October 2007 (UTC)

Multi-tiered Peer Review

No, think not on the word "bureaucracy"; I have not come to establish levels of reviewers. Instead I come bearing the idea that there could be 5 or so levels of reviews labeled simply A through E (nothing too descriptive.. that would be a headache).

The point is to make the process of reviewing more well-defined & less intimidating.

Level E, the easiest, would be just three or four thingies.. spellcheck, I dunno what else. That way newbies could swim in the kiddie pool, non-initimidating, etc. Some points:

  • I don't think articles should be allowed to request a review level; everyone will ask for A so that they can get as much bang for the buck as possible.
  • Different people can offer different levels of review on the same article: I could do an E review, then Awadewit or LaraLove or SandyGeorgia could swoop in and do an A on that same article, if they desire.

There are only 2 requirements placed upon reviewers:

  1. When you review, you explicitly state the level offered.
  2. For any level of review offered by a reviewer for a given article, you must make a good faith effort to provide all the reviewing aspects required for that level.

The only trouble, of course, would be coming up with the right mix of clearly spelled-out requirements for each level.. but y'all are smart... trust me, you can make it (that line gets laughs in Taiwan; long story). Cheers! --Ling.Nut

Interesting idea but I'd prefer to see reviewee and reviewers actually working on the article together to improve it. Thus it becomes less of a review and more of a seminar. My take is that the influx of new editors would iron out wrinkles swiftly. This already happens informally of course but PR could be were refocused (repurposed) more specifically towards cooperative effort. I'm not sure I like the idea of Wiki gate-keepers, by the way. --ROGER DAVIES TALK 15:23, 16 October 2007 (UTC)
Yeah of couse. That would be part of the process (wondering why anyone would think it wouldn't). Moreover, where are the wiki-gatekeepers in my idea? I must have made it sound completely different than the way it appears in my head. --Ling.Nut 15:29, 16 October 2007 (UTC)
Sorry if I misunderstood you. I have seen many peer reviews which aren't hands on. They're a list of tasks for the reviewee to complete. I'm advocating writing hands on specifically into the process.--ROGER DAVIES TALK 15:49, 16 October 2007 (UTC)

(undentage). OH that's what I left out! I don't mean the checklist should be something that reviewers should place hideous green check or red X images beside; no no no. Reviewers should adhere to the wisdom of Viva {{sofixit}} and actually fix shtuff. The E level reviews probably wouldn't involve much give 'n take between reviewer and dedicated editor (I don't like the term "custodian"; reminds me of that guy on "One Day At a Time" ... Schneider..) simply 'cause there ain't much to do in the first place, so it's hard to split the work. But higher-level reviews would be more teamish. --Ling.Nut 15:56, 16 October 2007 (UTC)

I think a move towards a more cooperative type of review sounds great. I think trying to put a level (A-E) adds an unnecessary extra level of (yes!) bureaucracy to it. One of the nice things about informal review is that it starts with one person fixing a couple of paragraphs, and it ends up being three or four people doing a wholesale rewrite - you never know where it's going! But the general approach sounds good. Walkerma 03:27, 17 October 2007 (UTC)

Next steps with the brainstorming list

I'm not sure what the best next step is after thinking about the criteria. I think one step needs to be that someone from this group picks an option and develops draft text to define it and propose it to PR (or wherever else it would need to be proposed). For example, if Ling.Nut wants to propose the "use a soapbox" idea, he would write up a more detailed justification, create a draft message to propose this at PR, and also perhaps a draft message to Jimbo's talk page. He'd put those in a section here or a subpage for this group to review, comment on and improve. Obviously others could work with Ling.Nut on drafting that if they were motivated too.

However, I think the current idea list is too long for us to do that; I'd like us to get down to no more than three concrete ideas and propose those in more detail. How do we get to three? Another straw poll? Do we need any more information on the items on the list, or are they all self-explanatory (or at least refer to discussions on this page that provide enough context)?

How about this as a way forward: we leave the brainstorming section open for another day or so, so that people can get any further clarifications they want, or post opinions supporting or opposing them (as I have done above for a couple of them). Then we open a strawpoll section, copying the list down and asking people to support or oppose, limiting themselves to no more than two supports. We leave that running for 48 hours. After that we pick the top three and some subset of those who support each of those three put together a subpage showing how they'd pursue it in detail. A few more days for discussion (no more than two or three days, I'd think, and less if we get to a quick consensus) and then we have a section here on this page to pick a course of action and then we go do it, whatever "it" is.

Does that sound too much process? I'm trying to lay out a clear (and fairly rapid) path forward, but I think we need patience and consensus and I do think some process is necessary. I hope that by focusing on this aspect of the workshop, I can let others discuss the ideas productively, while keeping things moving. Let me know if I'm interfering too much with the natural flow of this, but I really think we need some kind of process coordination, and that's what I'd like to provide. Mike Christie (talk) 18:03, 16 October 2007 (UTC)

Specific suggestions were provided here and then here (note in the latter the former is discussed with some agreement). But perhaps we're tripping over eachother with "fix it this way."
The brainstorming list is now too diffuse for a proper poll on it. I'll repeat myself from yesterday: do we want to make tweaks to PR or do we want to overhaul PR? I'd start with a yes/no strawpoll on that. (Or perhaps we've already decided?) Marskell 19:14, 16 October 2007 (UTC)
I think more than cosmetic changes are desirable, so that would be a vote for overhauling rather than tweaking. That's what I was trying to say with the first criterion above. However, I think my top choice would still be to find ways to provide kudos to reviewers, so that might be a tweak in your eyes. My second choice would be something from the "Wording changes, clarifications and reorganizations of existing structure" section above; perhaps the idea about making PR happen on article talk pages.
Clearly the next step is to winnow the list, but would answering your question (tweak vs. overhaul) really do that? Could we identify ideas with little support and remove them? Seems like that's just a straw poll working from the bottom up, instead of the top down. Mike Christie (talk) 22:40, 16 October 2007 (UTC)
What about picking one or two small things and one medium/large thing? Awadewit | talk 22:47, 16 October 2007 (UTC)
It would be an either/or on the question of overhauling, but it would not rule out tweaking. Either we're really rejigging it or we're not. We can still tweak in the meantime. Indeed, as the weaker of the options, tweaks are sort of assumed. Marskell 22:49, 16 October 2007 (UTC)
I should really just not comment when I'm ill. Logic escapes me at these times. :) Awadewit | talk 22:56, 16 October 2007 (UTC)
I'd be more inclined to tweak PR first as I don't think the whole system is broke enough to warrant investing huge amounts of energy with an overhaul will require.cheers, Casliber (talk · contribs) 02:55, 17 October 2007 (UTC)
I'm inclined to agree with Casliber. I think we just don't have enough reviewers and sometimes reviewers don't know how to review very well. I think there should be a place where you can ask "so, what do you think?" That is basic writing advice - ask someone else to read over what you have written. It is good, I think, that it is independent of any "grading" system where editors might be more concerned about "getting the grade" than working on revising the article. Awadewit | talk 04:05, 17 October 2007 (UTC)
Isn't that exactly what the Peer Review is supposed to be? An independent assessment without grading? --Malleus Fatuarum 04:11, 17 October 2007 (UTC)
Yes, it is supposed to be that way right now, but if it were folded into GA, etc. that would disappear. Awadewit | talk 04:16, 17 October 2007 (UTC)
Obviously that's true, and for that reason it shouldn't happen. But it equally ought to be made clear that PR isn't a part of wikipedia's hierarchical classification system. --Malleus Fatuarum 04:26, 17 October 2007 (UTC)
To take my preferred, if PR became a list of editors to contact for particular subject areas the "independent assessment without grading" would not be lost. As Jayron noted, it would cease being a place to "drop off" articles and become a place to "pick up" editors. Overhauled but not discarded. The barnstar and how-to would still be possible. Marskell 07:38, 17 October 2007 (UTC)

(undent) Yes but. Yes but if I understand your suggestion correctly, everyone will contact SandyGeorgia, LaraLove, Awadewit etc. It'll be a beauty contest. Some people will be flooded with requests; others will be sad, drooping wallflowers. Yes but it's passive as far as the reviewers are concerned: If I'm a new reviewer then I just put my name on a list and wait for someone to contact me (and wait...and wait... and...). If no one contacts me, then I feel I'm not an active part of the system, then I quit. Yes but... it seems inefficient, for reasons I'm finding hard to articulate... Am I off track here? --Ling.Nut

No, you're not wrong. This is what I was getting at when I spoke of Wiki-gatekeepers. I think any system based on specific individuals is a bad idea. There's a major risk of creating cliques/cabals of editors who share very similar values and who cross-fertilize (if you'll excuse the expression) only with each other. This already happens to some extent and editors whose ideas have been adopted at PR often feel protective of the article as it progresses up the hierarchy. I'm not sure what the solution to this is. --ROGER DAVIES TALK 08:02, 17 October 2007 (UTC)
I'm flattered. I'm usually the sad, drooping wallflower myself. :) I do understand these concerns, but I also wish there were a better way to connect editors with the same interests or editors willing to review particular subjects. I would like to be able to easily contact people interested in the eighteenth century, but, alas, there is no category for us (I had to make my own userbox). Perhaps some sort of separate listing of editors' interests, not associated with PR specifically, would be a good idea? Barring that, I think arranging PR by subject matter is a good idea. I know I review more GA articles because they are organized that way. Awadewit | talk 08:48, 17 October 2007 (UTC)
"I also wish there were a better way to connect editors with the same interests or editors willing to review particular subjects." While I understand Roger's concern, I think what Awadewit notes is more urgent. We've got a situation where nearly one out of two receives no comment beyond a bot. I can't think of any realistic way of getting people to sign on beside User talk; if no one contacts you, is there any real harm? Marskell 09:17, 17 October 2007 (UTC)
I wasn't actually thinking of you (sorry!) but of military history articles. There we have "hands on" reviews and people pitch in. We also run nag messages on the project talk pages if an article is under-reviewed. The down side is that it gives milhist editors a vested interest in promotion. In one recent FA, Issy Smith was promoted almost entirely on the basis of support from milhist members. It would have been better to get wider input (from say WPBIOGRAPHY and WPAUSTRALIA) but this didn't happen. In fact, milhist reviews are pretty robust but for FA the review needs to be much broader. One of the problems with "vested interest" reviews is that they can easily lose focus on broader issues, like whether the material is accessible to the general reader (for instance, the kid looking for information for an assignment). I suppose the answer is to move reviews under the umbrella of projects, with transclusions to other interested projects.--ROGER DAVIES TALK 09:26, 17 October 2007 (UTC)

(undent, and edit conflict) Hmmmm. Answering marskell: Well GA arranges articles by topic; not reviewers. This is extremely acceptable. Folks can informally gravitate to their area of interest. Absolutely no harm in that, and probably much good. The point is, that under that arrangement reviewers choose the articles they review, rather than dedicated editors choosing the reviewer they prefer.... do you want either sytem? Both? --Ling.Nut 09:33, 17 October 2007 (UTC)

(also edit conflict, tch!) This idea can work - whilst some reviewers will inevitably be in more demand than others, as long as there's a maintained list of participants somewhere there's nothing to stop those reviewers passing on requests to others with less of a workload. If the more experienced reviewers could also keep an eye on newer reviewers to ensure quality, I'm sure it wouldn't be long before they too will find the requests coming in. Initially this may put the burden of PR on a few experienced people, but it has the advantages of eventually widening participation whilst keeping up the quality, and will be self-winnowing; poor reviewers will either be guided towards improving their output or will be quietly dropped as the requests dry up. Neither does it (or should it) exclude any other interested parties from commenting on an article - in fact, this would be a useful recruiting ground as specific invitiations to join the PR project could be issued based on such 'outside' input. EyeSereneTALK 09:47, 17 October 2007 (UTC)
  • The idea of it makes me nervous, for reasons I and Roger davies gave.. But I'm not so strongly attached to my dissenting view that i would stand in its way.... let's assume you can persuade others to agree & move forward on it... my question in that case would be, if you change it in that manner, how long before you revisit the change in order to assess its success? If the new system is not an improvement, how long do we let it continue unexamined? --Ling.Nut 10:40, 17 October 2007 (UTC)

I may be in a minority here, but I thoroughly enjoy participating at peer review when I have time (when is that, LOL). Because there is no perceived "deadline" or time pressure or need to follow up right away, I like to dig in there. At PR, I'm more inclined to go ahead and chip away at the MOS issues when the nominator is friendly, and find that PR turns into a long-term relationship with the article and the nominator, as they often come back to consult through GAC and FAC. At FAC I'm more inclined to say the nominator should do the fixes themselves; there are too many other articles at FAC to work exclusively on one while ignoring others because it's such time intensive work. Anyway ... I'm wondering if a setup similar to WP:1FAPQ would work at PR, where there's just a signup list of people willing to do X, Y or Z type of work at PR (recall how 1FAPQ had a list of people willing to ce, etc.) I'd really not like to see PR arranged by category as GA is; that choppiness and overhead is a particularly frustrating part of the GA pages for me. I go to PR, scan the entire list, and pick a couple of articles that tweak my interest, regardless of topic. It's fun! I wouldn't want to scan by category as that is restrictive, and top heavy processes just make everything harder. The thing that needs to happen at PR is to bring articles that are languishing with no input to the attention of Projects or interested editors, and that can only happen if PR develops a culture or a WikiProject that reaches out to bring the reviewers in. Do all Projects have announcement templates like WP:MED does? Each Project should have a method of watching for articles at PR, and nominators should be encouraged to sign up there. SandyGeorgia (Talk) 02:04, 25 October 2007 (UTC)

Attempted summary of main current ideas

I didn't see much support for a straw poll to winnow the ideas down, so here's my take on the top three ideas right now. The third one on this list is my own idea and hasn't been discussed much in the last few posts, but I think it had enough support further up the page to include.

  1. Make PR into a list of articles to be reviewed. No reviews happen at the PR page; reviewers click through to the article and leave a review at article talk.
  2. Make PR into a list of available reviewers. Editors wishing to get an article reviewed can post at user talk if they wish.
  3. Use barnstars or medals based on activity and quality to provide kudos and encouragement to reviewers.

I'm reluctantly going to leave out Malleus's suggestion of putting stars of some kind in article mainspace to show GA or PR has happened, since Ling.Nut posted in agreement that this was unlikely to be productive.

OK, if the above list is reasonable, I have a couple of comments and a suggestion.

First, we need a method that prevents links aging into inaccuracy. For example, if article links are left on the page, they shouldn't have to be removed by the article editor since otherwise many will age indefinitely. If user links are left, those users may go inactive and then editors would be posting to inactive user pages. I don't think this kills the ideas above, but it makes the user-based idea a little harder since some form of cleanup would be needed (perhaps by a bot).

Second, I don't think the first two ideas actually encourage reviewer participation. They may be better or more natural organizations for a PR page, but I don't see how either would increase reviewer activity directly. Hence I'd suggest that we actually formulate a proposal combining some kind of barnstar or medal with one of the first two suggestions.

Here's a candidate merged version. I've tried to add in some of the mechanical details which would help eliminate some of the weaknesses I mention above.

  • To request peer review on an article, an editor simply adds a template to the article talk page; perhaps something like "{{pr-requested}}". A bot, running about twice a day, would write a dated {{la}} listing to the peer review page.
  • Reviewers would provide the reviews on the article page. Perhaps they could be asked to use "Peer Review" in the title of the section, perhaps. GA currently has an "on review" tag to avoid multiple simultaneous reviews; there'd be no reason to require this but a reviewer working on an article could certainly add an "under review" tag to the article talk page if they wished. They would not remove the article from the PR page when done.
  • The bot would remove the article from the PR request list after some period; I'd suggest 14 days. It would also, of course, remove the {{pr-requested}} tag from the article talk page. When doing so, it would count editors who had contributed to the article talk page other than by the PR-requester, and log those. (This could be limited to editors who'd posted under a "Peer Review" section title if we think that's better.)
  • The bot log would be used monthly to generate lists of PR-reviewers by number of articles reviewed. If we find this is subject to misuse because people are posting very short reviews just to get high on the list, we can try to fix that problem when it happens; I suspect it won't happen much because it would be a lot of work to get yourself on that list.
  • If we want to, we could issue a barnstar to the top one, or five, or ten people on that list.
  • Finally, in addition to the above, a PR users page could be established. To get on it you simply add something like a {{pr-reviewer}} tag to your user page; a bot creates the PR users list daily from those tags. The bot also checks contributions and if you've been inactive for 30 days you are not included on the list.

The bot could also remove PR requests that have been posted to GAN or FAC.

One downside of having the reviews occur at article talk is that you can't see, as a possible PR reviewer, which articles have had no attention yet. Is there a way around that?

-- Mike Christie (talk) 12:36, 17 October 2007 (UTC)

  1. "Reviewers would provide the reviews on the article page." In mainspace??? Isn't this the most flaming self-reference imaginable?
  2. Why so many bot activites? Plain old categories can keep track of PR requests & of PR members.
  3. How're ya gonna make a bot smart enough to know who has reviewed what? Scrape the talk pages and look for... what? Comments? Methinks there is .. well at least I see no need whatsoever for bots here. --Ling.Nut 13:46, 17 October 2007 (UTC)
Oops, meant to say article talk space, per the original suggestion. Sorry. If bots aren't needed, great; I think eliminating stale requests, at least, is likely to benefit from being done by a bot. But they're not intrinsic to the idea. How to make the bots smart enough--how about this: if reviewer X has the pr-reviewer tag on his user page, and X edits a talk page while it's got the pr-review request tag on it, X gets credit for a peer review contribution. Mike Christie (talk) 16:10, 17 October 2007 (UTC)
I see that this doesn't treat things as mutually exclusive, which is good. "They may be better or more natural organizations for a PR page, but I don't see how either would increase reviewer activity directly"—this makes you sound non-committal, Mike. Do you not think that dropping user talk notes will increase reviewers? Note, the reviewers would sign up for a category.
FA has a reviewing barnstar: Image:Reviewer's Award.png. Can't say that's increased reviewing though. A converse example is the WP:DYK medals—I know those are a hit so that's what we should try to emulate.
Lastly, I'm not anti-strawpoll; I just wasn't sure about having one that covers twenty issues at once. Marskell 18:03, 17 October 2007 (UTC)
It's not that I'm non-committal; I do think that user talk notes are more likely to increase reviewing activity in some ways. However, not everyone who occasionally reviews at PR will be willing to have their name listed, so I think it's likely the pool of apparently available reviewers will actually shrink somewhat, offsetting the increase in their responsiveness. The net effect might indeed be positive, but I think we should try to find other ways to help encourage reviewers as well.
On barnstars, I have some comments below in response to some of Awadewit's very reasonable reservations, so I'll post down there instead of here. On the strawpoll, you're probably right, but it's kind of hard to shepherd a group like this into consensus at the right speed, and I was concerned we wouldn't focus if something like a strawpoll didn't happen. Maybe the strawman merged proposal I started this section with will do the job of focusing the discussion. Mike Christie (talk) 01:10, 18 October 2007 (UTC)
I must say that I am pretty opposed to the idea of awarding barnstars based on the sheer number of peer reviews that someone has done. I think that would encourage short reviews (something like this happened at the GAC backlog reduction drive, IMO) rather than thorough reviews. It would also not encourage the kind of cooperative reviewing that many of us have been asking for. I think the barnstars should be awarded by individual editors to peer reviewers for a "job well done". We could advertise them prominently, but doling them out based on quantity rather than quality seems to defeat the purpose to me. Awadewit | talk 20:53, 17 October 2007 (UTC)
Interesting! I have been proposing a similar restructuring of GAN recently, i.e., nomintation and review happens on the talk page, using talk page templates, and a bot to maintain a central list and track the review process. It would be nice if the mechanisms of GAN and PR were similar, since it would make cross-over of reviewers easier.
A bot is needed: there is a limit to what you can do with categories. Watching categories provides no information about the changes to category entries. In addition, there is no dating mechanism, so it would be impossible to remove PR requests older than 14 days (unless the category was watched for changes... by a bot) which would probably generate a demotivating backlog of old requests.
Furthermore, I think a central page, listing requests, generates a sense of identity, and replacing it by categories would probably also be demotivating. I share the concerns raised that counting edits might encourage multiple brief comments, and drive by peer reviewing, rather than a comprehensive analysis. One possibility is to count bytes instead. This is actually not much harder, since both involve querying the talk page edit history and parsing the list of names and dates: parsing the byte count could be done at the same time. But I do appreciate Ling's point that we should not overuse bots. Geometry guy 21:25, 17 October 2007 (UTC)
  • I definitely like the idea of peer-reviewing happening on the talk page of an article. Clicking back and forth is annoying! It would also make the reviewer feel "closer" to the article, I think, and perhaps even more likely to help out with it, furthering our goal of cooperative reviewing.
  • Could we at least organize the requests by categories? I missed the part where we were replacing the list with just categories. (I think the GA system works pretty well here, except it has too many subcategories.) Awadewit | talk 00:00, 18 October 2007 (UTC)
  • I am not sure that counting bytes is a good idea for peer review awards, either. I generally copy sections of text from the article into my reviews - it would be ridiculous to count that, but the bot can't differentiate (I don't think) between that text and my comments. Also, if a review descends into some sort of petty war over nothing that spills into hundreds of kb, that shouldn't be rewarded. I still think awards are best given on a personal level with personal messages. Perhaps I am a minority in thinking awards with personalized messages are more meaningful? Words gets around about who is a good reviewer anyway, I have found. Awadewit | talk 00:00, 18 October 2007 (UTC)
Awadewit, I agree with everything you say in principle; I guess I have been wondering if there is some value in treating these awards or lists as social engineering. If we were all sitting in offices in the same corridor, and reviewing each other's work by walking down the corridor for a chat, the social reward in the form of thanks would happen as a natural part of the communication. My suggestions about lists and awards and so on were an attempt to find a way to bypass the need for personal messages, since not everyone sends messages.
If there's no support for the idea of a list of active contributors, then we should drop it. I was hoping to develop a sense of friendly competition, but it may not be possible; the FA list, for example, is something I keep a slightly embarrassed eye on, and would like to move higher up on. As for a barnstar, and personal messages to go with it, I think it can only do good, is presumably easy enough to create.
For your other points, I think categories on the page are a good idea so long as the individual articles up for peer review can still be seen. Are we all agreed that article talk is the best place for the review to take place? And that PR noms should disappear after some period of time? And that users who are inactive should not appear on any list of users who are available to do peer reviews? Mike Christie (talk) 01:10, 18 October 2007 (UTC)

(undent) man it's hard to track these threads!

  1. if there is a membership list, then not only do i see no reason to cull it, I think culling it is counterproductive. It irritates me when people remove me from something i didn't really wanna be removed from; perhaps others feel the same.. Moreover, what harm is there in leaving a name on a list? If someone asks for a review from an inactive member, then let that inactive person decline. If that person doesn't wanna be a reviewer any longer, let that person remove his/her name (takes two seconds). The only thing you might do is put a prominent notice somewhere that if someone doesn't answer a review request in (say) three days then you should w/draw the request and try someone else. Conclusion: No bots needed to review a list that shouldn't be culled.
  2. Barnstars: you make make a humble-looking one for quantity of reviews and a prettier one for quality. Or.. skip that idea and just make one pretty one, and let people use it in any manner they wish.
  3. PR requests older than 14 days can be culled by any WP:BOLD PR member. Requests should be dated.. as they are in GAC. No need for bots, yet again.
  4. I think I'm done now. Lunch time. Fish head soup, anyone? --Ling.Nut 05:53, 18 October 2007 (UTC)
(1) I agree entirely. (2) That is certainly one option. I also agree with general remarks that counting bytes may not be much better than counting edits. GAN has a semi-official coordinator who awards a barnstar to the reviewer of the week. PR could do the same. I'd still like to keep an open mind about using bot-generated data to inform such a coordinator about reviewer activity, but agree that the personal touch is much better when making awards. (3) This is where a bot could help to keep the PR process entirely on the talk page, by autogenerating the PR request lists. Categories would be easy to implement using something similar to the GA "topic" parameter in the talk page templates. GAN manages without a bot, but not very well, in my view, because reviewers still have to make multiple trips to the GAN page to add "on review", "on hold", "second opinion" and to remove the article. As Awadawit notes, this is annoying. I also do believe PR requests should be removed from the list after a fixed period. The talk page template could be replaced by an answerphone-like message "I'm sorry no reviewer was available to answer your request... please try again later" maybe with a link to a clip from Vivaldi's Four Seasons :-) Geometry guy 09:07, 18 October 2007 (UTC)

Sorry again to be late to this, but while I agree in principle, I see lots of issues on implementation. The {{articlehistory}} template currently "stores" peer review information in history; if they occur on article talk pages, that's lost. I'm not sure it matters so much where the review happens, and there are advantages to having it in a separate file. I think encouraging more reviews is more important than the "where". SandyGeorgia (Talk) 02:11, 25 October 2007 (UTC)

Attempting to organize recent ideas (again)

P1

Proposal: Peer reviews should take place on the talk pages of articles.
Comments:

  • I haven't seen any disagreement with this proposal yet (but there is a lot of discussion going on). It seems like an excellent idea, as it would promote involved reviewing. Awadewit | talk 06:30, 18 October 2007 (UTC)
  • Dig it. The Peer Review page should STILL exist, and should STILL contain a list of the article requesting review, but having the comments on the article talk page is far more helpful for fixing the article, involving more people, and it keeps the discussion closer in Wikispace to where the actual work is done, which is preferable. --Jayron32|talk|contribs 06:31, 18 October 2007 (UTC)
  • Support. Makes sense in terms of watchlists and closeness to the reviewing process. Marskell 08:19, 18 October 2007 (UTC)
  • Support - no good reason to do otherwise, and lots of good reasons to try it. EyeSereneTALK 10:42, 18 October 2007 (UTC)
  • Support - per EyeSerene --ROGER DAVIES TALK 10:48, 18 October 2007 (UTC)
  • Support, per all the above. Mike Christie (talk) 12:33, 18 October 2007 (UTC)
  • Support, it's an excellent idea. --Malleus Fatuarum 19:03, 18 October 2007 (UTC)
  • Yes, I agree. I would like to see the entire process take place on article talk. Geometry guy 19:21, 18 October 2007 (UTC)
  • Pile on Support. --Ling.Nut 17:03, 19 October 2007 (UTC)
  • oops. Completely disagree. This will cause us to lose peer review info as a file/page in articlehistory, and I'm not convinced that it solves the issue of not enough reviewers. I'd much rather see movement in the direction of encouraging more reviewers without so radically altering the process. And as soon as you move to talk pages, info gets broken up; there are advantages to having one file of PR info. At FAC, we can often review a peer review and note items that weren't addressed; much easier than trying to work through an entire talk page. SandyGeorgia (Talk) 02:14, 25 October 2007 (UTC)

P2

Proposal: Peer review requests should be categorized on the peer review page, but the names of the articles should still be visible (much like GAC is now).
Comments:

  • I also think that this is a good idea - there needs to be some organization! (But then, I am a compulsive organizer. I have all of my books entered into a database.) Awadewit | talk 06:32, 18 October 2007 (UTC)
  • Again, I dig it. More organization would help potential reviewers find articles in their expertise and interest. This seems like a good idea, though we MAY not want to be as detailed in out PR categories as at WP:GA, where the catagories are almost out of hand. Having maybe a dozen or so categories in broad terms (the arts, history, science, etc.) may be best.--Jayron32|talk|contribs 06:34, 18 October 2007 (UTC)
  • Yes, but use the FA not the GA categories. Marskell 08:24, 18 October 2007 (UTC)
  • Request for clarification: is this proposed as an alternative to editors contacting individual reviewers for a review, as a parallel nomination method, or merely for tracking purposes (or any/all of those)? EyeSereneTALK 10:40, 18 October 2007 (UTC)
  • I think that it is parallel at this point. Awadewit | talk 10:54, 18 October 2007 (UTC)
  • Parallel in the good sense, I think. We can do both. Marskell 15:26, 18 October 2007 (UTC)
  • Thanks Awadewit and Marskell for the clarification ;) I'll add my support too (might be worth linking the nomination category to the appropriate section of the reviewer list if the list ends up being organised that way). EyeSereneTALK 16:32, 18 October 2007 (UTC)
  • Support. I agree the GA organization may be more than we need. FA categories would be OK, but I don't think we have to use them. Somewhere else on this page someone suggested that it could be a parameter in the nominating template; that would work but has risks such as mis-spellings and so forth. If people place their article templates directly on the PR page, they can put them into the right category. However, this has been a source of slight errors in GA, since it's been fairly common for someone to leave the article name as "Your Article" rather than substituting. It would be nice to figure out a fix for this; GA has a bot that reports on it but doesn't fix it. Mike Christie (talk) 12:41, 18 October 2007 (UTC)
  • Support. The full GA organization is certainly more than we need, but the top level of the hierarchy would be ideal: these 11 topics are equivalently the Wikipedia 1.0 categories, and are already wired into several templates such as {{GA}} and {{ArticleHistory}} as the "topic" parameter. They are therefore readily available and familiar to many editors. The same parameter could be used in the proposed {{pr-requested}} etc. templates. Such consistency would save work, as the topic parameter would only need to be set once, when the article first meets some sort of content review. As far as I can see, the FA categories are only used on the featured article page itself, not during any process. It would be nice to harmonize them with the 11 topics, but that is another story. Geometry guy 19:42, 18 October 2007 (UTC)
  • Pile on Support. Can wrangle about which topics to use, but last time I looked the GA topics were getting a bit out of hand. G-guy's comments above sound about right G-guy's bad-hand sock again -- Ling.Nut 17:06, 19 October 2007 (UTC)
  • Sorry. Don't like the category idea, and don't like top-heavy processes; they discourage rather than encourage participation. I enjoy participating at PR precisely because it's fun and I don't feel I need to restrict my input to any category, and I really dislike the categorization at the GA pages. SandyGeorgia (Talk) 02:17, 25 October 2007 (UTC)
  • PS, there's been a lot of movement with no input from Gimmetrow; are we prepared to undo a lot of {{articlehistory}} ? GAC and GAR can be passed by one editor and have no file in articlehistory, which is frustrating; peer review currently does have a file and can always be found. SandyGeorgia (Talk) 02:19, 25 October 2007 (UTC)
Huh? Last time I looked, GAC & GAR were both options in the {{articlehistory}} template. Dr. Cash 02:25, 25 October 2007 (UTC)
They have no archived page; sometimes they may have a talk page link, wihch is most often hard to find. *Every* FAC and FAR has a page, which is archived, as does *every* PR. There is no such thing on GAC and GAR, and when trying to correct and rebuild articlehistory GA errors using Dr pda's script, one often has to spend a long time fiddling through history and diffs to find the GA event. This is taking a weakness of GA relative to other processes (no separate archivable page) and implementing it in a process which has always had an archived PR page. SandyGeorgia (Talk) 03:05, 25 October 2007 (UTC)
  • Support I agree with using the top-level categories to sort PR nominees by. It would make it much easier to find articles that we're interested in or qualified to review. It would also be easier to help deal with backlogs; for example, if there's a huge backlog on some of the arts or history nominations, but not so much of a backlog in geography or science, we can direct our backlog review requests and announcements towards editors primarily interested in those categories. Dr. Cash 02:22, 25 October 2007 (UTC)

P3

Proposal: There should be a list of peer reviewers, listing their interests, that editors can contact.
Comments:

  • I, personally, would love such a list. I think it make it easier to connect reviewers with editors, but I understand the reservations others have. I wonder if separating the list from the peer review page itself a little, as GAC has done (they also have such a list), might work. Awadewit | talk 06:29, 18 October 2007 (UTC)
  • Like the idea, and like Awadewit's addition to it to have the list on a separate page. It almost goes without saying, too, that the list should be voluntary and optional, and that anyone at anytime should be able to review an article.--Jayron32|talk|contribs 06:43, 18 October 2007 (UTC)
  • Yes, and these could be included on the main PR page (if not, new editors may miss it). Point not just to reviewers but clearly active projects, as well. Marskell 08:23, 18 October 2007 (UTC)
  • Support (like the idea of including relevant projects too). I realise there are objections to rewarding by number of reviews, but is it worth including this type of data on such a list (eg number of reviews undertaken, active/inactive etc)? EyeSereneTALK 10:34, 18 October 2007 (UTC)
  • Support. I'd like to keep these on the same page, if it doesn't get too unwieldy a list. One possible organization would be (a) active Wikiprojects with review processes; (b) active reviewers willing to be contacted (perhaps with a one sentence summary of their interests, if they are willing to review only in certain areas; and (c) articles, sorted into categories. Mike Christie (talk) 12:46, 18 October 2007 (UTC)
  • This all sounds good to me. All on the same page sounds fine. If any of the lists become unweildy, they could either be linked, or put in show/hide templates. Geometry guy 19:46, 18 October 2007 (UTC)
  • Have expressed fears about this proposal; cannot Support or Oppose until I see how ... aggressively... it is pursued. Switching to a system in which choosing a reviewer is your only option is something I would Oppose. A parallel system.. in which the option to choose a reviewer is de-emphasized... and adding one's self (sometimes spelled oneself; dunno why exactly) to the reviewer's list is wholly and completely optional... and all in all the whole thing just doesn't get out of hand darn it! .. might maybe be somewhat OK-ish. Did I hedge enough? --Ling.Nut
  • Support, similar to the list at WP:1FAPQ, but will only work if a PR culture develops, and that would need to be encouraged. SandyGeorgia (Talk) 02:20, 25 October 2007 (UTC)

P4

Proposal: There should be some sort of automated process for adding and removing peer review requests as well as peer reviewers.
Comments:

  • Maybe, but I think there should NOT be bot-generate or script generated reviews. These are never helpful, and somewhat rude. If you don't have the time to review, IMHO, it is better to say nothing than to leave an unhelpful, script generated generic review. However, certain basic automated tasks, like clearing reviews from the page after 1 month may be a good idea. Also, this may be able to be combined with the above, in that we can have a bot that buzzes people's talk page when a review appears that matches their interest. For example, lets say that I have an interest in reviewing sports articles. When a new article gets added to the sports list, a bot could (if I request that it does) drop a note on my talk page asking me to review the article. How does that sound? --Jayron32|talk|contribs 06:39, 18 October 2007 (UTC)
  • I'm also a maybe. We need to flesh this out more. Marskell 08:25, 18 October 2007 (UTC)
  • I don't know enough about how bots work to really have an informed opinion on this matter. Awadewit | talk 10:55, 18 October 2007 (UTC)
  • This doesn't have to be automated, but I think it's going to be a pain in the neck to manage by hand, and it seems easy to automate. The "stale reviewer" removal seems more important than "old pr request" removal; if we do talk page interaction it's going to be quite discouraging, especially for new editors, to get no response to requests for assistance. Both are important though. I guess my question to those who oppose this would be, first, do you think it is necessary to remove (or at least flag) inactive reveiwers, and to remove articles from the request queue after some period; and second, do you think a human would be willing to do that maintenance? Re Jayron's question about banning bots -- I think bot reviews should be done on request only, but I'd suggest we avoid making that proposal this time round since it's a negative change rather than positive. Mike Christie (talk) 12:51, 18 October 2007 (UTC)
  • Depends on what "some sort of automated process" means. It's certainly a pita to have to keep switching around in the GA review process to update tags/article history on the GA project page and on the article talk page, and that manual process is error-prone. If this proposal is suggesting some kind of a server-side process that could centralise and validate that process then I'd be in favour. Otherwise probably not. --Malleus Fatuarum 20:03, 18 October 2007 (UTC)
  • Managing the central pages in content review processes is a pain. Passing a GAN right now involves updating (1) article talk, (2) the GAN page, (3) the GA page, (4) the GA number. I hate closing GAR discussions, because it requires a similar amount of pointless to-and-fro. If the GAN and GA pages were autogenerated from article talk templates, there would be just one stop: article talk. The same is true for PR. The nominator simply adds {{pr-requested|topic=...}} to the article talk page, and the article name is automatically listed in its category by a bot. A reviewer finds it there, or is contacted individually, and everything else takes place on the article talk page. So, I am all in favour of using a bot here, and am certainly willing to flesh it out: although I don't write bot scripts myself, I know how they work, and have experience from the Maths 1.0 project, which automatically generates lists of assessed articles from a talk page template. Geometry guy 20:07, 18 October 2007 (UTC)
  • Split this proposal into two or three! I'm maybe-ish on automagically removing requests... it sounds kinda peremptory to me, but it might be a necessary evil.. and Very No on removing members. The other bot stuff that Malleus Fatuarum and G-guy discuss above is an intriguing possibility. if it can work, it may very well be a Good Thing. --Ling.Nut 17:18, 19 October 2007 (UTC)

P5

Proposal: There should be some sort of reward system for peer reviewing. Options include a list of top reviewers, such as WP:WBFAN; barnstars for the reviewers who review the greatest number of articles; barnstars for the reviewers who offer the most thorough reviews (as counted by kb); or just a general encouragement of personal barnstar giving.
Comments:

  • I think that personal barnstars should be the limit. There are too many problems with counting the numbers of reviews (encourages short, uninvolved reviews) and counting the bits/bytes (what would be counted, exactly?). Awadewit | talk 06:28, 18 October 2007 (UTC)
  • Again, I agree with Awadewit. If someone notices that another editor is doing a lot of work at PR, a personal barnstar may be a good idea. Also, if there is a short-duration drive, like a backlog elimination drive, with moderators watching it, and there are "barnstar prizes" given for that one-off event, that may be OK, but having a quid-pro-quo system of automatically awarding barnstars for reaching certain benchmarks seems like a bad idea. --Jayron32|talk|contribs 06:42, 18 October 2007 (UTC)
  • I'm not as opposed as others to counting the number of reviews and awarding medals on that basis. Again, using WP:DYK as an example, you find a healthy sense of competition with such processes. Perhaps it could be number of reviewers where the initial nominator indicates they received good feedback. Marskell 08:27, 18 October 2007 (UTC)
  • I agree with Marskell; I think we're worrying about a problem we don't know will happen, and the potential upside is worth a try. However, I also agree that a barnstar is a good idea. Mike Christie (talk) 12:53, 18 October 2007 (UTC)
  • Personal awards certainly. I'm neutral on the issue of count-based medals, but I agree there is probably no harm in trying it. Geometry guy 20:26, 18 October 2007 (UTC)
  • Have someone make a spiffy barnstar or two. Maybe make suggestions about barnstar usage, but no rules. Everyone should work together to engineer a supportive and encouraging environment, with psoitive feedback for good work, etc etc. --Ling.Nut 17:22, 19 October 2007 (UTC)
  • Personally, I don't like the award idea, and don't "work for awards"; I'm concerned about quick drivebys and the quality of some of the GACs that pass during sweeps. Others may work for awards, so I'm not opposed. I do a peer review because and when it's fun, but I 'spose other editors may have other motivations. SandyGeorgia (Talk) 02:24, 25 October 2007 (UTC)

[Note: Please adjust proposals if you feel they are in error and do not reflect the conversation. Thanks. Awadewit | talk 06:13, 18 October 2007 (UTC)]

General comments

  • Erm... as I see it.. and I could be wrong.. but to me it looks like if you simply copy the GAC format in its entirety, it solves most of the probs listed here:
  1. Articles arranged by category.
  2. Reviews take place on Talk.
  3. Again, removing reviewers is a bad idea IMO.
  4. Requests are dated (they aren't removed if old, but could be).
  5. The only thing missing is a member list with interests given.. that actually provided elsewhere I think but not directly on GAC or GA. --Ling.Nut 06:20, 18 October 2007 (UTC)
  • I was kind of thinking it might be easier to comment on each proposal individually. Sorry for not making that clearer. Awadewit | talk 06:28, 18 October 2007 (UTC)

Ling.Nut makes a very valid point nevertheless. GAC is already doing most of what's being talked about here. All that's needed is for PR to adopt what GA is already doing well and to remove itself from the wikipedia article hierarchy. Problem solved. --Malleus Fatuarum 20:09, 18 October 2007 (UTC)

I agree, and am glad you make that second point again, because it is an important one. PR should not be part of the hierarchy: it should be available to any article at any stage.
Nevertheless, I think it is a good idea to make the mechanisms of PR and GAN similar, since this makes cross-over of reviewers easier. I also think that if a PR reviewer believes that the article they are reviewing meets the GA criteria, they should list it as GA, just as they would recommend nomination at FAC if they thought that was appropriate.
However, I am glad we are looking at each issue individually rather than simply aping GAN, since we have an opportunity here to make something that works better. In particular, there is too much administrative work in GAN at the moment because almost everything has to be done twice, once on article talk, and once on the GAN page. I think that should be fixed, not copied. Geometry guy 20:21, 18 October 2007 (UTC)
I agree with all of that. --Malleus Fatuarum 20:26, 18 October 2007 (UTC)
  • I'm confused - what do you mean that PR should "remove itself from the wikipedia article hierarchy"? Awadewit | talk 20:17, 18 October 2007 (UTC)
I understand this to mean that PR should not be thought of as a "stage" e.g., between GA and FA, but something that is a separate process that can be applied to articles anywhere in the assessment hierarchy. I believe it is, but it still tends to be used primarily to feed FAC, and I think it would generate more activity if it were more widely used. Geometry guy 20:23, 18 October 2007 (UTC)
That's exactly what I meant, thanks for the translation. :) --Malleus Fatuarum 20:30, 18 October 2007 (UTC)
Oh, yes. I thought we all agreed on that. Awadewit | talk 20:47, 18 October 2007 (UTC)
Hm. The above reforms reinforce to me that, let's say, a semi-merge with GA would be very logical. You would nominate for PR (simple review) or for PR + GA (review plus confirmation that it's a "Good" article). But that can wait until the next Current topic... Marskell 21:06, 18 October 2007 (UTC)
I just think that this is a bad idea. There should be some place you can go for help without being "graded". Awadewit | talk 21:10, 18 October 2007 (UTC)
I didn't suggest otherwise. Marskell 21:19, 18 October 2007 (UTC)
I think I'd likely be a very strong supporter of that idea. But probably best to wait, as you say. --Malleus Fatuarum 21:27, 18 October 2007 (UTC)
  • If PR copies the GAC format, I 'spose I'd not likely participate in PR anymore; I don't like the categorization and top-heavy processes. If it works for others, I won't oppose, but I am seriously concerned that Gimmetrow hasn't been part of this discussion, as it affects articlehistory. SandyGeorgia (Talk) 02:27, 25 October 2007 (UTC)

Nitty-gritty details

It looks like we are really getting somewhere! I think it is time to try and iron out exactly what our proposal is.

P 1

We want peer reviews of articles at any stage to be done on the talk pages of the articles.

P 1 and P 2

We want peer reviews to be listed at a centralized PR page.

P 2

We want peer reviews to be categorized at the centralized PR page. What is not clear is what categorization scheme should be used. Possibilities include: FA, and GA top-level (or equivalently Wikipedia 1.0).
More discussion is needed here:

  • I think the GA-top level categories are a good choice. Since not all articles can become FA, this might provide some uniformity. I know less about the 1.0 categories. Awadewit | talk 20:47, 18 October 2007 (UTC)
  • GA-top level categories and Wikipedia 1.0 categories are the same (which is another reason why it is a good choice): I've reworded the above to reflect this. (See e.g., {{WP:1.0}} and Wikipedia:Version_0.5.) Geometry guy 21:55, 18 October 2007 (UTC)

P 3

We want a list of peer reviewers, by interest. It is not clear whether or not this should be included on the same page as the peer review requests themselves.
More discussion is needed here:

  • I would suggest a prominent link, otherwise the page will be too long. Look at GAC already. Awadewit | talk 20:47, 18 October 2007 (UTC)
  • I'm somewhat against listing reviewers against a rather crude set of categories. And definitely against the idea if it's a one-to-one relationship. For perhaps the majority of PR reviews knowledge of the subject area is likely to be irrelevant anyway, given that it isn't a hierarchical review. --Malleus Fatuarum 21:02, 18 October 2007 (UTC)
  • This word "hierarchical" is really distracting to me for some reason! Perhaps "evaluative"? I actually think that reviewers' knowledge of the subject area is quite relevant. There are certain subjects that I would never review, because I have so little knowledge of them that I could not provide much help at all. Sports comes to mind immediately. I would have no idea if the page was well-researched (what are best sources for soccer players? I have no idea), comprehensive, or well-organized. Well-written I might know something about, but even that is difficult, because each discipline has its own language. Copy editing sports pages is a nightmare for me because I don't know the jargon. I agree that reviewers should not necessarily be tied to only one category and not always tied to that category, but I think that many of us would find such a list helpful. What do others think? Awadewit | talk 21:09, 18 October 2007 (UTC)
That word "hierarchical" is at the root of the problem with PR. Euphamism or neologism don't solve that problem. --Malleus Fatuarum 21:18, 18 October 2007 (UTC)
  • I'm sorry, I should have been clearer. I didn't mean for PR to use that word, I meant for you to use that word. I think it would be easier to follow your criticisms of the current PR process if you didn't use the word "hierarchical" and used the word "evaluative" or some such synonym. I must say that I don't really see this as a problem as PR at all. Most editors asking for reviews post comments like "This article is 'start' and I would like to get it to GA - please help me" or "This article is GA, please help me get it to FA" or some such. I have just never gotten the feeling that PR has the kinds of problem you are outlining. That does not mean we shouldn't try to make it as open as possible, but I think we might be trying to solve a non-problem. Could you maybe outline in a little more detail what you see as the exact problem? Thanks. Awadewit | talk 21:35, 18 October 2007 (UTC)
  • I have attempted to outline what I see as the problem with PR on a number of occasions now. I have no intention of continuing to shout into the wind. We clearly have to agree to disagree. --Malleus Fatuarum 21:44, 18 October 2007 (UTC)
  • Actually, it is not clear to me at all what you see as the major problems at PR at all and I just spent time rereading every single one of your posts on this page. I'm not really sure I'm disagreeing with you, because I'm not really sure what you are saying. I'm sorry if you think I'm being dense, but in an environment where everything is written, it is best to spell out one's ideas as clearly as possible, in my experience. Awadewit | talk 21:55, 18 October 2007 (UTC)
If I can attempt to translate again, I think Malleus means that if PR is not part of a hierarchy of review processes, then its reviewers should be generalists. If that is the case, then I disagree. For me, the watchword for PR is flexibility. A peer reviewer can approach the article as a generalist or an expert: all input is welcome. There is no particular goal other than improving the article. I think it would be useful to list reviewers and their interests, but I agree with Malleus that it would be a bad idea to pigeonhole reviewers into the same categories as articles. Geometry guy 22:32, 18 October 2007 (UTC)
  • I don't think anybody was suggesting that peer reviewers should only review one category of articles, were they? I too, agree, that reviewers should be able to review as generalists or as specialists. The list would help those interested in specialist-reviews. Awadewit | talk 22:36, 18 October 2007 (UTC)
Agreed. Geometry guy 22:41, 18 October 2007 (UTC)

P 4

We want automation of some sort. Possibilities include: bots that alert subject area reviewers when new articles appear in their categories; removing inactive reviewers; removing PR requests that haven't been filled; some sort of bot that reduces the work of the GA-system.
More discussion is needed here:

  • I like the idea of bots alerting reviewers, if that is possible. I don't like the idea of being removed (unless the user has been indefinitely banned). I think maybe a 30-day waiting period is more realistic than a 14-day waiting period at this point. Whatever automation reduces the clicking around, I am in favor of. :) Awadewit | talk 20:49, 18 October 2007 (UTC)
  • Automation that reduces the work involved in the GA-system, yes. But I don't see the need for subject area reviewers to be notified, or indeed the need for there to be subject area reviewers at all for that matter. Unless we want PR to masquerade as something that it doesn't claim to be. --Malleus Fatuarum 21:13, 18 October 2007 (UTC)

Digression on specialist vs generalist peer review

  • ...I don't see the need for subject area reviewers to be notified, or indeed the need for there to be subject area reviewers at all for that matter. Unless we want PR to masquerade as something that it doesn't claim to be. --Malleus Fatuarum 21:13, 18 October 2007 (UTC)
  • I don't think it would be a masquerade to have people with some knowledge of a topic reviewing articles about those topics - it would be a benefit to the article. If you have ever had someone review an article about which they know nothing, you will realize why this can lead to disaster. Please try to imagine reviewing an article on a topic about which you know nothing. Could you really do a good job? This doesn't mean there shouldn't be reviewers who provide a "lay" perspective (I often do this for science articles myself), but articles must be reviewed at some point by people who know what they are doing, otherwise there is no real way to tell how good the article is. For example, if I tell you that I have used the best Mary Wollstonecraft scholarship to write the nine articles on her and her works that I wrote, how do you know that is true? You could easily find some article or book that says something different about Wollstonecraft than the articles I wrote. However, I spent a long time culling the best sources for what major scholars agree upon. Only someone else who has done this would really know that. Otherwise you have to trust me. In some ways, this is one of the fundamental problems of wikipedia. It is often very difficult to know how good someone's research is, if you yourself do not know the field. That is one reason why it is good to have reviewers who know something about the subject, who can say, "wait! you're missing this major theory!" Awadewit | talk 21:47, 18 October 2007 (UTC)
  • I'm afraid that your reply simply confirms to me that you have not taken on board the idea that PR is not a peer review. Hence, I suppose, your earlier confusion about "hieracrchy". --Malleus Fatuarum 00:15, 19 October 2007 (UTC)
  • It is not an academic peer review, but that is something very different entirely, which relies on credentialed experts. When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers? Awadewit | talk 00:23, 19 October 2007 (UTC)
  • Quite. So my access to your sources of knowledge about what you consider to be the "best" scholarship on a subject such as Mary Wollstencraft is not relevant to my ability to conduct a PR review. Unless you consider PR to be a part of the article ranking hierarchy of course. --Malleus Fatuarum 00:33, 19 October 2007 (UTC)
  • Your use of the word "hierarchy" here is continually confusing and I can barely parse your sentence. I can't see that you would have any access to "sources of knowledge about what I consider...." There are no sources about what I personally think. I used sources that are considered the best, demonstrably so. I simply do not understand why you think PR is "hierarchical" in any way. Articles gain nothing in the wikipedia "hierarchy" of grades by going through it, unlike GAC, FAC, or A-review. Awadewit | talk 01:09, 19 October 2007 (UTC)
  • Sorry, I forgot to answer the last part of your question; "When you have submitted articles to peer review, what has been your experience with generalized vs. specialist reviewers?" Are you asking me about real life or on wikipedia?--Malleus Fatuarum 00:37, 19 October 2007 (UTC)
  • I am only talking about wikipedia PR. Awadewit | talk 01:09, 19 October 2007 (UTC)
  • (undent) Awadewit and Malleus Fatuarum: I hereby officially threaten (yes, it's a threat) to give both of you one of those sappy wiki-smiles or goofy wiki-cookies if you don't stop being unhappy about the fact that neither of you understands the other. :-) PR can offer both generalist and specialist reviews. 'Nuff said. In my humble opinion, it might be useful for folks to always overtly/explicitly/prominently mention which type they offer. But if I say that too loudly, everyone will scream the "B" word (bureaucracy)... --Ling.Nut 03:09, 19 October 2007 (UTC)
I agree with Ling here (but then, he is my bad hand sock, so of course I would :-) that PR can offer both generalist and specialist reviews (and anything inbetween); it should be flexible. In my view, Malleus is using the word "hierarchy" in two different senses: the Wikipedia article quality hierarchy, and the hierarchy of knowledge. This is confusing. We seem to be agreed that PR should be separate from the article quality hierarchy. For me that strengthens the case that peer reviews can be as general or as specialized as the reviewer chooses. I can elaborate if necessary, but this topic has already received plenty of discussion. Geometry guy 17:45, 19 October 2007 (UTC)

Discussion of automation continues below

  • I'm very wary of automation. I especially strongly dislike the idea of removing inactive editors from some mebership list via bot. Membership lists are always and everywhere voluntary anyhow!! Plus.. I've argued above... I don't see the bot-craziness as a good thing in most cases. Let's start with the idea tha that "The fewer bots, the better!" and have only those functions tthat are really necessary and really better performed by bots be done by them. Done talking. Coffee time! --Ling.Nut 03:09, 19 October 2007 (UTC)
Why? I agree that removing editors from lists is a bad idea, but it makes no difference whether a bot or another editor does it: it is still a bad idea. I don't see any bot craziness here, and you make no arguments to support your position. Ah... I get it now: you are setting up a straw man to make it easier for me to make my case :-) No, seriously, I have in mind that there would only be one bot: the question is, what tasks would it perform. In my view, the main work-saving task, and the one which seems to receive the most support elsewhere, is to maintain (and in my view, entirely autogenerate) the PR main page. I suggest starting from that position, and then applying your principle to other tasks that we might consider automating. Geometry guy 17:53, 19 October 2007 (UTC)
PS. I hope you don't mind me breaking your comment to restart the thread.
  • (undent). I don't mind anything, except the crazy schizophrenic organization of this page.grumble grumble
  • I don't like bots 'cause they're cool. I'm not joking. Cool things scare me. Again, I'm not joking. They scare me 'cause everybody wants to be the bright girl or boy who comes up with a new bot to organize this, delete that, soon we'll have a barnstar bot counting bytes and awarding barnstars, then we'll have a bot adding people to the membership list (resistance is futile - you will be assimilated), then we'll have bots making reviews (which already happens.. I send out lotsa wiki-love to whoever invented that bot and whoever has used it, but stop now please, 'kay?)... I said it earlier.. and I meant it.. use bots if and only if they make a meaningful contribution in a way that is really really really a pain to do manually. Plus i have seen some folks express concerns that bots gobble up resources... --Ling.Nut 18:21, 19 October 2007 (UTC)
Well, I agree that bots should be used only where necessary, and that bots should be simple. Bots have a control setting which ensures that they don't use up resources, but the kind of bot I am proposing will not even need that, because: (1) It will only write one or two pages, perhaps once or twice each, per day; (2) It will never, ever, download a page!! Geometry guy 18:47, 19 October 2007 (UTC)

P 5

We want some sort of reward system, including perhaps all of the following: personal barnstars and barnstars awarded by quantity of reviews (to be determined by a bot somehow - either number of reviews or kb).
More discussion is needed here:

  • Do we want to try all of the above barnstars and see what works best? Awadewit | talk 20:49, 18 October

2007 (UTC)

  • These discussions have ended up rather fragmented, but I'll shove this in here ;) Personal barnstars are a great idea - it might be nice to have more than one type (eg an 'official' PR project award, a 'slap on the back' type award from other reviewers, and one that can be awarded by article nominators to show appreciation to particularly helpful reviewers). I don't believe awarding by number of reviews is the best way to go though. I believe that anything that even inadvertently puts quantity over quality is not good. However, if kudos and a sense of friendly competition do need to be generated, listing the number of reviews along with the reveiwer name (as I proposed somewhere above) could meet this need (maybe a bot could handle this). As an aside, it would also help nominators pick an active reviewer from the list. If consensus goes the way of award per review count though, it might be fairer to do this by hand rather than based on a bot, as then quality can also be taken into account. EyeSereneTALK 17:46, 19 October 2007 (UTC)

Question and comments

I'm lost. (Though your attempts to organize have been great, Awadewit.) Can we go through P1 to P5, one-by-one? Marskell 21:10, 18 October 2007 (UTC)
  • Sure, why don't you lay it out. Awadewit | talk 21:47, 18 October 2007 (UTC)
OK! Marskell 21:57, 18 October 2007 (UTC)
This already is laid out I think: try this. Geometry guy 22:04, 18 October 2007 (UTC)
No. The original poll was edifying—this sub-thread is just unspecifying what had been specified. Marskell 22:11, 18 October 2007 (UTC)
Okay, I've sub-subthreaded it, in case it is still useful, but we can move on if you like. [[User talk:Geometry guy|Geometry Marskell 22:50, 18 October 2007 (UTC)guy]] 22:19, 18 October 2007 (UTC)
I think this thread is officially nuts. (Though the original poll was a great act of clarification.) Marskell 22:21, 18 October 2007 (UTC)

(unindent) Actually, I never intended any of them to be a poll. If you notice, I marked "comments" and "discussion" under everything. I thought it would be a good idea to separate the discussion on each topic. It was getting all jumbled up. I was surprised when people started writing "support". That was not what I thought was going to happen. Awadewit | talk 22:27, 18 October 2007 (UTC)

But it was a good thing it did happen. We should not be anti-poll. We should be anti too many discussions at once. Marskell 22:32, 18 October 2007 (UTC)
I think Awadawit did a great job there. In my view, the reason people started writing "support" was because of this: the essence of our common view had been distilled. This is a great place to move forward from. Let us make the best of it! Geometry guy 22:35, 18 October 2007 (UTC)
Yes, let's—material available. This thread now has thirteen headlines and would be impossible to understand other than by the ten odd people who have already commented. Scrub what I wrote in the next thread, but still come up with something focused on a given point of agreement and how it might be implemented. Marskell 22:40, 18 October 2007 (UTC)
That's actually exactly what I was trying to do, hence the title "nitty-gritty details". Sorry I wasn't more clear about that. Awadewit | talk 22:44, 18 October 2007 (UTC)
No, it's all of the details at once that I'm not understanding. We're just going to get lost in talk. We winnowed and now, weirdly, we're broadening again. But whatever. Bedtime here. Marskell 22:50, 18 October 2007 (UTC)
I don't think it is broadening, but isolating the precise proposal we want to make takes time and, because it is all written out, painstaking work. Good night! Awadewit | talk 22:52, 18 October 2007 (UTC)

I've unindented the thread. I am neutral on how to proceed, but I think the thread already contains some useful info. Geometry guy 22:57, 18 October 2007 (UTC)

I see I came down rather hard on this thread! Sorry. I was a little bleary-eyed. Yes, it does contain useful info already, so we can consider it as we move to each point. Marskell 06:47, 19 October 2007 (UTC)
Well it is rather fragmented and has some irrelevant digressions, and I think it is a good idea to proceed point by point. Unfortunately, the thread below is developing digressions as well. I guess there is so much consensus around, that we have to digress to find something to argue about ;-) Geometry guy 17:57, 19 October 2007 (UTC)
  • Not sure where to park this comment, but I'm generally not on board with most of the proposals above. I don't see the advantages of moving PR to talk pages when the core problem is not *where* the PR happens, but encouraging more reviewers. A top-heavy, fragmented process like GAC will not encourage more reviewers, and I'm discouraged that PRs would no longer be in a file, accessible in articlehistory. Of course, I could be entirely wrong on everything; maybe other reviewers would do more PRs if they could get an award, but that's not why I do them. I won't oppose any changes proposed here, but I'm not likely to participate in a PR process categorized like GA is. SandyGeorgia (Talk) 02:30, 25 October 2007 (UTC)

Accepted: Peer reviews should take place on the talk pages of articles

OK, thinking "methodical but organic" we should probably take each of the consensus points one at a time. (Anything mentioned in the above strawpoll is still just tentatively accepted, of course.) So people are going to start PRs on talk pages. What do they do? Let's pretend we're all two-year olds here and need to be led through it. Should threads have a set title to help, say, a bot doing a review? What templates needed and where to place them? Marskell 21:57, 18 October 2007 (UTC)

Why not do it the way that GA does it? --Malleus Fatuarum 00:25, 19 October 2007 (UTC)
GA uses the {{la}} template. A nominator enters this, substituting for ArticleName:
# {{la|ArticleName}} ~~~~
which turns into this text (as viewed in edit mode):
# {{la|The Nightmare}} –[[User:Outriggr|<font color="#112299">Outriggr]]</font> [[User talk:Outriggr|''§'']] 07:32, 6 October 2007 (UTC)
which actually displays like this (except that the links work):
The Nightmare (edit|talk|history|links|watch|logs) –Outriggr § 07:32, 6 October 2007 (UTC)
So GA does get the review to take place on the article talk page, which is what we want to achieve for PR too. The good thing about this way of doing it is that the nominator gets to pick the category that the article will be nominated in. The bad thing is that it is a bit prone to errors. Take a look at the bot exception report, for example, which shows three current malformed nominations (at the time of writing, anyway). This isn't an unusually high count -- see this historical report, for example. None of these particular errors are really important, but they indicate that it's not particularly easy to follow the instructions. Occasionally people will put a nom at the top of a list, instead of the bottom. Another negative is that to do a nom you have to make two edits; reviewing, passing, and failing are also a little time-consuming. Geometry Guy gave the reassessment page of GA as an example of a page that's a pain to work with.
Ideally, if it can be done, I would think we'd only want humans to do those things that humans are needed for. That would be requesting a peer review, and performing the review. Ideally, again, I would think we'd like those to be done with a single edit, preferably without having to add anything to a template that could allow typos.
I think the simplest way to make all these things happen is to have a template with no parameters placed on the article talk page by the editor. Everything ought to happen from that. There's no need to require the requesting editor to sign or date the request; and if a bot can work from those articles with templates to create the page, that would be ideal. The bot would update the PR page, listing the articles using some abbreviated version of the {{la}} template. To review, you'd just click through to the article. There would be no requirement to title your review with any particular heading; you'd just write whatever you wanted to write.
There are some things this does not address:
  • Categories. This could be an optional parameter. If not, or if the param were blank, could the bot guess the category from any wikiproject banners on the page?
  • Visibility of which articles have had reviews and which have not. I don't see how to do this without having peer reviewers provide some tag as well, in their peer review, to be searched for by the bot and reported on the PR list page.
Then after some period (Awadewit suggested 30 days above) the bot would remove the article from the PR page and edit article talk to make the PR tag historical.
Marskell, is that the sort of thing you were looking for? Mike Christie (talk) 01:26, 19 October 2007 (UTC)
I think you need to distinguish between complexity for the nominator and complexity for the reviewer. In my view the GA nomination process is simple enough to be suitable for PR, but it's a bit of a pita for the reviewer, which is maybe where some changes could be made. And I really have failed to understand this recurring requirement for bots to remove anything automatically after how ever many days.--Malleus Fatuarum 01:47, 19 October 2007 (UTC)
[Query: I saw you use "pita" this way before and I just assumed it was a typo - I guess not. As I love learning new words, could you explain this one to me? I looked it up, but all I got was what I knew and obviously not what you are referring to: plants and bread. Thanks. Awadewit | talk 02:02, 19 October 2007 (UTC) ]
(edit conflict with Jayron) Re the requirement to remove things automatically; it's not the "automatically" that I feel is important, but the removal. The important one is the inactive reviewers; I think people who want to request a peer review should have some way of seeing that a particular potential reviewer hasn't edited for a month or more. For articles, PR does currently have a rule about removing them after some period. All I'm suggesting there is automation of that sometimes tedious task. Mike Christie (talk) 03:05, 19 October 2007 (UTC)
  • Perhaps that's what I don't understand. In what way is a request for a peer review linked to the activity or inactivity of any particular reviewer? --Malleus Fatuarum 03:34, 19 October 2007 (UTC)
  • Number 1) Are we now at the point in this process where we want to seek further comment before proceeding? It looks like we have a pretty good idea of what we want to do, and it may be rude to drop all of these changes at PR without at least notifying the people who DO work there that we are doing it ahead of time, and seek their input? Do we also need to notify the Village Pump at WP:VPR as well?--Jayron32|talk|contribs 03:02, 19 October 2007 (UTC)
  • Number 2) As far as categorizing goes, consider the way that AFD handles it with the "preloaded debate". It is a really spiffy thing, and if we can find who coded that, maybe we can get them to help out with this. The Preloaded Debate thing has really neat instructions, very clear and detailed, and it automates MOST of the important, but easy to mistype or screw-up stuff. The Preloaded Debate seems to head off many of the errors people used to have with AFD, especially with categorizing mistakes and the like.--Jayron32|talk|contribs 03:02, 19 October 2007 (UTC)
  • I don't like the idea of a "pre-loaded" review. It's not "spiffy", it's insulting to the editors. --Malleus Fatuarum 03:39, 19 October 2007 (UTC)

Re Jayron's point, yes it is probably time to at least make a note on PR itself. I don't want to go back to the VP until we've hammered out more details but we can maybe rope in a few more people.

Re template. Imagine that each of the categories we have on the PR has a designated shortcut. So I have an astronomy or astrophysics article; I check the shortcut and add:

  • {{la|ArticleName|astro}} The last parameter automatically categorizes it under the appropriate heading.

Taking the idea that GA and PR might be semi-merged, I could also do:

  • {{la|ArticleName|GA|astro}} This would automatically generate it on both the PR and GA pages.

If the editor is making an error in either case, the template will point it out. Or is this asking too much of our templates? Marskell 06:59, 19 October 2007 (UTC)

I disagree about notifying PR next, because I disagree that we should implement our ideas as soon as we have a consensus. Reason: the issues that we are addressing are interlinked, and we may need to modify some of our ideas in the light of future topics. I suggest instead that we update the mainspage page of this workshop to indicate proposed solutions. Geometry guy 07:06, 19 October 2007 (UTC)
I'm in strong agreement with Gguy here - the last thing we need is to alienate everyone by jumping in too soon on project pages with fixes that will be debated, (possibly) agreed and implemented... and then changed in the light of further debate here. There is still much to discuss. EyeSereneTALK 11:04, 19 October 2007 (UTC)
I agree. I think we should wait until we have a very specific proposal to offer. The devil is in the details. Awadewit | talk 11:22, 19 October 2007 (UTC)

I haven't seen any concensus that GA and PR should be merged. Quite the oposite in fact. The suggestion is that PR could adopt some of what GA currently does well, not for any kind of merger. --Malleus Fatuarum 07:16, 19 October 2007 (UTC)

OK, we won't let PR know yet. Now then, how about these templates? Marskell 18:20, 19 October 2007 (UTC)
whew, close call. Are you all convinced this is really adopting "what GA currently does well" and that others will perceive same? SandyGeorgia (Talk) 02:57, 25 October 2007 (UTC)

I like the idea of doing article reviews on the article talk page, instead of on a separate page. However, there could be a potential issue with archiving. When talk pages are archived, it may not be obvious to the editor doing the archiving that the link to the PR review in said talk page needs to be updated as well if the {{ArticleHistory}} template is used (this is a problem I've noticed with GA, since GA reviews are done in article talk as well). So this system may result in more errors if we don't address the issue with {{ArticleHistory}}. I think it's quite important for the reviews to be easily accessible so that the review history can be found as the article is continually improved. Perhaps the thing to do is to copy the peer review to it's own archive, stored in a sublisting under the article's talk page, like, 'Talk:Article/Peer Review 1', then link to that page. It might be good if GA did this as well. Dr. Cash 21:19, 19 October 2007 (UTC)

Yes, thank you for noticing. PR in an archive is an important distinction from having to go digging through pages of talk page history to find GA reviews. PR is clearly linked to a file in articlehistory, while finding GA info usually requires a time-consuming search. I'm still not sure what was perceived as the original problem that necessitated moving PR to talk pages. How does that, and adding "process", encourage more reviewers? SandyGeorgia (Talk) 02:57, 25 October 2007 (UTC)

Returning to the topic

(New section to avoid curtailing the two digressions above, while returning to the topic here.)

Here are a couple of quotes from the above, which I would like to comment on.

Mike: "The good thing about [the GAN] way of doing it is that the nominator gets to pick the category that the article will be nominated in. The bad thing is that it is a bit prone to errors." and "I think the simplest way to make all these things happen is to have a template with no parameters placed on the article talk page by the editor."
Malleus: "I think you need to distinguish between complexity for the nominator and complexity for the reviewer."

Okay, so my question to Mike is: why no parameters? We are agreed that the nominator should get to pick a category for the article. So the way to do that is to have a mandatory topic/category parameter in the template. This touches on Malleus's point in a couple of ways. In my view there should be an entry level requirement for all content review processes: the nominator should have to do something to get the ball rolling. The problem with the GAN approach, is that when the nominator gets it wrong, it creates trouble for reviewers, who have to put it right, so complexity for the nominator and the reviewer are not always distinct. However, with a talk page template, if the reviewer gets it wrong, the article doesn't get listed, the template adds the talk page to an error category, and it is up to the nominator to realise this and fix it.

I don't see any point in making this topic/category parameter optional, and it isn't feasible to get a bot to add it. Please remember, if we want someone to code a bot for us, it had better be simple, and that means sticking to the number one rule of simple bots: they never, ever, read a page. All they read is categories and "what links here". They also write their output to a page from time to time. I'm not saying this just to make life easy for the bot: following this principle actually leads to well-designed automation, without unnecessary complexity. Geometry guy 18:28, 19 October 2007 (UTC)

As said above: Imagine that each of the categories we have on the PR has a designated shortcut. So I have an astronomy or astrophysics article; I check the shortcut and add:
  • {{la|ArticleName|astro}} The last parameter automatically categorizes it under the appropriate heading.
If the editor is making an error in either case, the template will point it out. Marskell 18:56, 19 October 2007 (UTC)
I haven't entirely understood: {{la}} is a general purpose template. Your example does the following: ArticleName (edit | talk | history | protect | delete | links | watch | logs | views). I guess you have in mind using a different name but achieving a similar effect? If I have understood correctly, there is a duplication here. and a source of errors: the template would have to be put in the right place on the PR page, and have the corresponding parameter set correctly. There is a lot of error checking that templates can do, but they can't check their location. Geometry guy 19:51, 19 October 2007 (UTC)

Okay, now to address the visibility issue, here is a second principle: once an article talk page gets a content review template of some sort, it retains some sort of content review template for the rest of its days. What this means, is that all editors/reviewers have to do is change the name of the template (keeping the topic parameter unchanged). In particular, an article which has had a peer review, will retain a template stating this, analogous to {{GA}} and {{DelistedGA}}. Once an article has been through more than one process, the content review information gets folded into an {{ArticleHistory}} template (preferably by GimmeBot to avoid errors, but that is another story).

The current article history format does record archived PRs. Marskell 18:57, 19 October 2007 (UTC)

Hand held guide: first try

Here's a first try at describing the process.

Guide for nominators

  1. Read instructions (like these ones) on the PR page, and figure out which category your article is in.
  2. Add the template {{PRrequest|topic=name of category}} to the article talk page.
  3. Wait.

If there is a list of reviewers, then the nominator can also be given the option to leave an invitation on a reviewer's talk page.

Guide for reviewers (version A)

  1. Look at the list of requests on the PR page and pick one you would like to review!
  2. Go to the article talk page and replace "PRrequest" by "PR" in the template (if necessary).
  3. Review the article and add your review to the talk page.
  4. If the article has now had a thorough peer review, replace "PR" by "PRdone" in the template.

I've chosen template names fairly arbitrarily. Also the PR template may not be needed, as multiple reviewers are to be encouraged at peer review; still it might be useful to distinguish between PR requests which already have one reviewer. One could also imagine using a PRmore template to indicate that a review had been carried out, but that further review was still needed. The basic model, though, is completely simple.

The templates would add the talk page to categories such as "Topic peer review requests", "Topic peer review requests under review", and so on. The bot would read this categories and use them to write the PR page once or twice a day with a categorized list of requests, annotating those under review, or those needing further review. If there is a list of reviewers the bot would generate this from a "what links here" to a template that reviewers place on their user page.

An alternative to the above, is that a PRrequest would run for a fixed length of time. The advantage of this is that the guidelines for reviewers become extremely simple.

Guide for reviewers (version B)

  1. Look at the list of requests on the PR page and pick one you would like to review!
  2. Review the article and add your review to the talk page.

After the fixed period, the bot replaces PRrequest by PRdone, although this could be done sooner by hand if the article has been well reviewed. This violates the simple rule of bots, in that the bot needs to read the talk page to do this (actually only section 0). However, it is a minor violation, and code for updating talk page templates by bots is widespread. Geometry guy 19:30, 19 October 2007 (UTC)

Now I'm confused because I don't see how #2 under nominators is different than my previous, though you've said it's a source of errors. (I was using {{la}} as an example only, because others had.) "There is a lot of error checking that templates can do, but they can't check their location." I had assumed the transclusion could proceed directly from the template. But if not, a bot will do it for us, as you're saying. I would skip "replace 'PRrequest' by 'PR'" as unneeded hassle. Finally, this seems to deemphasize seeking out people on user talk. The sit-and-wait approach is the thing we want to change. Marskell 08:17, 20 October 2007 (UTC)
{{la}} is placed on the GAN page, so it has to put at the right place on the GAN page. I assumed this was what you had in mind for your PR template. However, if you meant that your template is added to the article talk page, then my suggestion is identical to yours. But a bot is then needed to list the article on the PR page: templates can only transclude information onto pages where they are explicitly present.
I'm inclined to agree that the 'PR' step is unnecessary, and agree we should include seeking out reviewers on User talk pages. (My "Wait." was slightly tongue in cheek.) Geometry guy 09:54, 20 October 2007 (UTC)

(undent) There's already a special page for pages that link to a template via transclusion, nyet? Rather than gathering the links onto the PR page by bot, can the special page be transcluded into a section on the PR page...? Inquiring minds wanna know, and so do I... --Ling.Nut 15:20, 20 October 2007 (UTC)

OK I just tried it; the standard sytax for transclusion seems to provide only a link when pointing to Special pages... but can we live with a link? --Ling.Nut 15:37, 20 October 2007 (UTC)
Sounds clever, although I couldn't find the special page you refer to. Anyway, I think the answer is no. Also, I believe we want to know the date on which the PR request was made. This again requires a bot to keep track of things. Geometry guy 15:54, 20 October 2007 (UTC)
I did mean adding right to the article talk—sorry if I wasn't clear.
But I wonder if people's eyes are glazing at this discussion. Maybe we should try to get the general brush stroke down and move along:
  • "The nominator will add a template to the talk page. Ideally, this will immediately transclude the page to PR; if not, a bot will do so. The nominator will contact people directly through a reviewer list."
OK, we agree at this level of generality. And then what does the reviewer do? And so on. Say, five points of principle to match the P1 through P5 above and then announce the general intent. Marskell 21:36, 20 October 2007 (UTC)
Well, you asked for hand-held steps. If you want broad brush instead, that is fine. However, I think it is important to know whether agreeing with the talk page template idea implies agreeing with some sort of bot, as we don't yet have complete consensus on whether or how a bot will be used. Template X on page A cannot transclude information onto page B, unless page B transcludes page A. The only way page B can know which pages transclude template X is for the template to put these pages in a category or categories. Even then, the best that page B can do is list the contents of these categories, like this. It is impossible, for example, for page B to provide the date when the template was added to page A. Geometry guy 11:18, 21 October 2007 (UTC)
Sorry, I didn't mean to contradict myself. The big brush stroke is already done with the strawpoll discussion. I just meant that rather than solving every particular, let's agree in principle to how to do P1. So if it doesn't transclude directly we either accept a bot or accept the nominator will place the template twice, once on article talk and once on PR, yes? But when you wrote "Add the template {{PRrequest|topic=name of category}} to the article talk page" where is the category derived to? Marskell 14:34, 21 October 2007 (UTC)
No problem, and sure, although it will probably be different templates on article talk and PR, since they have to do different things. I'm not sure I understand the last question (what does "derived to" mean?). Maybe an example will help: the template{{PRrequest|topic=Arts}} will transclude [[Category:Arts peer review requests]] onto the talk page, and hence place the talk page in Category:Arts peer review requests. Does that answer your question? Geometry guy 14:52, 21 October 2007 (UTC)
Yes, I understand. My first thought, however: is it necessary? Do we need to categorize them? My second thought is that I don't want to continue to go in circles, so I'll borrow from Awadewit the successful create a proposals list under a new thread. OK? Marskell 16:03, 21 October 2007 (UTC)
The categories are needed for the bot to list the articles by topic on the PR page (unless we have a separate template for each topic, as Jayron suggests below). They are also needed for the bot-free approach. Geometry guy 16:26, 21 October 2007 (UTC)
Q: you told me templates by themselves can't transclude the article to a specific location... What's the difference between a general template plus parameter {{PRrequest|astro}} and a topic specific template {{PRrequest-astro}} Marskell 16:46, 21 October 2007 (UTC)?
The former is more adaptable and easier to maintain (as it is just one template in template space). The latter has the advantage that you don't need categories for a bot to work, because the bot can use "What links here" to the template, rather than read the category. This definitely requires a bot, though. There is no difference as far as transcluding information to a specific location is concerned. A parameter has the advantage that it meshes well with other content review templates, in particular {{ArticleHistory}}. Geometry guy 17:33, 21 October 2007 (UTC)

(Outdent) Actually, rather than starting a whole new thread, let's list them:

  • No templates: Start a thread on article talk, note the thread on PR under your preferred subject category.
  • A single template on PR: Start a thread on article talk and place {{la}} or something like it on PR. Reviewers will find their way to the thread.
  • A single template on article talk: Start a thread on article talk and place a template at the top of the page; a bot will place it on PR. At least one parameter will be needed unless we want the bot to generate an undifferentiated list. For instance, it could be placed in a category and the bot could read the category and update PR accordingly.
  • A single topic specific template on article talk: Same as last, but avoids the category and the bot.
  • A template each on PR and on article talk: See GAC.

Is this basically it? Marskell 16:23, 21 October 2007 (UTC)

Yup! Although the template on PR is a fairly cosmetic issue: it doesn't really do much apart from creating a consistent look for entries. Geometry guy 16:29, 21 October 2007 (UTC)

Topic specific templates

Why not make it idiotproof? Why not just have a series of topic-specific templates that generates the right topic in the PR request. So that we could have {{PRHistory}} and {{PRSports}} templates that autogenerate the correct parameters? We could list each template in a banner at the top of the page with instructions on how to use it? Ideas? --Jayron32|talk|contribs 19:04, 19 October 2007 (UTC)
There is some merit in this, but a parameter has the advantage that it can be carried through the entire content review process. Also, it can be made pretty robust. Compare:
{{GA/Topic|SocSci}}: Social sciences and society
{{GA/Topic|socsci}}: Social sciences and society
{{GA/Topic|Social sciences and society}}: Social sciences and society
{{GA/Topic|Social Sciences and Society}}: Social sciences and society
This is used by {{GA}} to fix common variations in category names. Geometry guy 19:36, 19 October 2007 (UTC)

Nomination procedure proposals

The following are means by which a nominator might list a peer review.

I think five editors agreeing and nobody dissenting is enough for consensus; I've added a version of Pr3 to the consensus section of the front page. I removed mention of the bot on the grounds that it's the method that will be used, and isn't intrinsic to the consensus. Please correct the statement on the front page if it's inaccurate.

Pr1

No templates: Start a thread on article talk, note the thread on PR under your preferred subject category.

  • Do two things? There must be a way to nominate an article for PR with one stroke. Awadewit | talk 20:50, 21 October 2007 (UTC)
    • Well, this would be two things with no extraneous considerations. Go here, go there--you don't need to know what "template" means. I don't imagine it could be so simple, but I thought I'd throw it out. Marskell 21:08, 21 October 2007 (UTC)

Pr2

A single template on PR: Start a thread on article talk and place {{la}} or something like it on PR. Reviewers will find their way to the thread.

Pr3

A single general template on article talk: Start a thread on article talk and place a template with a topic parameter at the top of the page; this will add the page to a category, and a bot will list it under the corresponding topic on PR.

  • What is the difference in level of difficulty technically-speaking between P3 and P4, since they are essentially the same from reviewers' perspectives (they have to find out the topic code or topic template)? If one is significantly easier to implement from a script-writing perspective, then I would support that one over the other. Awadewit | talk 20:38, 21 October 2007 (UTC)
They are sides of the same coin, yes. With one template (3) you have less overhead, less to make and watch. But you need the categories for the bot to watch. With multiple templates (4) you have to create a template for every category created on the PR page, but you don't need categories because the bot can read "what links here" on each individual template. So G'guy explained, anyhow :). Marskell 20:42, 21 October 2007 (UTC)
Yes, I get that, but what I'm asking is which is more difficult to set up from a wiki-code, wiki-template, etc. perspective? Which will require more expertise to run? I wanted to support the one that is easiest to maintain, for example, and doesn't rely on the knowledge of one person's master program. Awadewit | talk 20:47, 21 October 2007 (UTC)
There is very little difference between the two. From a reviewer's perspective, the difference between P3 and P4 amounts to the difference between
  1. {{PRrequest|topic=Arts}} (or {{PRrequest|Arts}})
  2. {{PRrequest-arts}}.
From a maintainer's perspective, I would say it is easier to maintain one template (P3) than several (P4). Also, the former is more adaptable, and fits in better with existing templates, such as {{ArticleHistory}}. But the difference is not huge and none of these templates will be very complicated. Geometry guy 20:56, 21 October 2007 (UTC)
  • At this point, then, I am leaning towards P3. Awadewit | talk 20:58, 21 October 2007 (UTC)
Ditto. As I posted, I was thinking as much. Marskell 21:06, 21 October 2007 (UTC)
I like this proposal the BEST of all of them, but I have a minor question: How does this work technically? That is, I know next to nothing about the opperation of bots. How does the bot find the article unless it searches every talk page; it seems like many requests could go missed... Just a technical question. If this is a non-problem, I definately support this proposal. --Jayron32|talk|contribs 17:37, 22 October 2007 (UTC)
I would assume it would use the list at Special:Whatlinkshere. CloudNine 17:42, 22 October 2007 (UTC)
I like this proposal too, but I'm concerned about the technical question that Jayron32 raised. Would the bot really have to search every talk page? How often would it be doing it? --Malleus Fatuarum 17:49, 22 October 2007 (UTC)
No it wouldn't, and never, respectively! All of the proposals adhere to the the number one rule of simple bots that I mentioned above: the bot never, ever, reads a page. In Pr4, as CloudNine suggests, the bot could use Special:Whatlinkshere to the various templates. In this proposal, to answer Jayron, there is only one template, and so Special:Whatlinkshere does not provide information about the topic. Instead, the template places the talk page in a category, according to the topic, and the bot reads the categories to find the PR requests and list them by topic. Geometry guy 18:25, 22 October 2007 (UTC)
That's good enough for me. Unequivocal support from me for this option then. --Malleus Fatuarum 18:35, 22 October 2007 (UTC)
I hope I'm not jumping the gun (but I think this is relevant from a coding perspective): would the same bot also handle delisting the article from PR once the template is removed? EyeSereneTALK 18:46, 22 October 2007 (UTC)
Once the template is removed (or replaced by a different template to indicate that the article has been peer reviewed), the talk page is no longer placed in a peer review category. Consequently the bot will no longer list the article on the PR page. Geometry guy 19:17, 22 October 2007 (UTC)
Ah, right - each time runs the bot rebuilds the entire PR page using the peer review category. I assumed it just added to/removed from it. Thanks! EyeSereneTALK 20:18, 22 October 2007 (UTC)

Pr4

A single topic specific template on article talk: Same as last; avoids the categories, but not the bot.

Am I right in thinking that if we implement Pr3, Pr4 becomes almost trivial, in that {{pr-review|arts}} could be called (if that's the verb I need) by {{pr-review-arts}}? If so, Pr3 permits Pr4 for those who prefer it. Mike Christie (talk) 21:20, 21 October 2007 (UTC)
Well, they don't permit eachother insofar as 3 is a single template that allows for different categorization, while 4 is multiple templates that would cover every category. They would accomplish the same thing, though. Marskell 21:33, 21 October 2007 (UTC)
If we implement Pr3, then yes, it is perfectly possible to provide {{pr-review-arts}} as a synonym for {{pr-review|arts}}, but there isn't a lot of point in doing so: this wouldn't really be Pr4, just syntactic sugar for those who can't find the pipe character on their keyboard ;-) Geometry guy 21:46, 21 October 2007 (UTC)

Pr5

A template each on PR and on article talk: GAN structure.

  • I find this tedious - there must be a better way. Awadewit | talk 20:40, 21 October 2007 (UTC)
    • I agree that asking the nominator to post one would be best (if that's what you mean). Marskell 20:44, 21 October 2007 (UTC)
      • Yes - that is what I mean. Sorry I wasn't clearer. Awadewit | talk 20:48, 21 October 2007 (UTC)

New thoughts

I have a general idea which I think could help solve the lack of reviewers problem, at least for PR and FAC, as well as some general comments. I may have missed it, but I did not see anything that really addressed the lack of reviewers above.

At WP:RfC, any new request for comment has to get two users to agree to it before it goes forward. I propose that new peer reviews or Featured Article candidates first enter a wait list. The nominator then has to go and review three other articles (I think three would work better than two, but it should be at least two). They then add the names of the articles they reviewed to their nomination, and once they have met the minimum number requirement, their nomination moves into the actual PR or FAC queue and can be reviewed by others. Outside reviewers would still be welcome, but they could not add comments to any article that was still waiting to enter the PR or FAC queue. This would combine the two main problems (lots of requests, lack of reviewers) and help solve both. I believe it would encourage only serious requests for PR and cut down on multiple simultaneous PR requests on closely related articles (as someone asking for 5 articles to be reviewed would have to first review 15 others). I also think this idea would help, as it is good to look at the work of others and this would force nominators to look at other articles.

For GAC there is just one review per article, so there a GAC queue could be established and to leave it they could either do one GAC review, OR some combination of three reviews for FAC or PR.

I think the easiest way to promote a nomination from the wait queue to the review queue would be to have any other editor besides the nominator be allowed to do this. They could look at the three articles and make sure there were substantive reviews by the nominator there. People receiving reviews could also comment on them in the waiting queue. This way if someone was a slacker and their three reviews were just "good job" or something similar, the recipients could note this and the request could go back into the wait queue, or not be promoted.

I think this system would actually work best for FAC, as the level of experience of editors nominating articles there is highest. On PR, perhaps it would be helpful to have a listing of what the person is asking for. I think three categories would be useful, the next step after this PR is either FAC, GAC, or general improvement. This could help all reviewers, but someone looking for general improvement could review other general improvement articles, etc. This would help match level of reviewers too (so a newbie would not have to review a future FAC).

Finally, I worry about making PR just a list. Now it is easy to see what articles have received no reviews, but I am not sure how easy this would be to see on a list only format. I hope this helps, Ruhrfisch ><>°° 14:05, 23 October 2007 (UTC)

These are all interesting thoughts, but I will just comment on the last point for the moment. It would be possible to show, in a list, which articles have received a review, provided the first reviewer modified the talk page template after making the review. I don't have a particular view on whther this is worthwhile or not. Geometry guy 19:25, 23 October 2007 (UTC)
Some sort of check mark, etc. could appear on the PR page. This is not an insurmountable problem, in my opinion, but Ruhrfisch is right to point it out. Awadewit | talk 22:13, 23 October 2007 (UTC)
I don't know what I think about requiring people to do peer reviews. I have a sinking feeling that it might generate poor reviews just so one's own article moves up in the queue rather than the opposite effect. However, I could be wrong - I'm just a pessimistic graduate student. :) Also, isn't it kind of against wiki-philosophy? All of these requirements? It doesn't sound as cooperative as I would like somehow. Awadewit | talk 22:13, 23 October 2007 (UTC)
I think the number one problem here is too many requests and not enough reviewers in PR, GAC, and FAC. I am also not thrilled with introducing a sort of wikitax or corvee, but really do not see any other alternative to get more people involved (nor have any been suggested above, as far as I can tell). I know for a while there was a suggestion at the top of the PR page that all editors submitting a PR request also review at least one other article. That did not work, and PR is barely working now (my best results are from asking people I know and some times work with to please look at my PR request, but that can be a bit incestuous). I guess I see this idea as a way of dealing with the free rider problem, or sort of forcing users to follow the golden rule (review unto others as you would have them review unto you). Since the goal of almost everyone requesting a peer review is to get the article to GA or FA, they are fine with dealing with those requirements, so what's the problem with also having PR requirements or requirements to enter the GA or FA process? As I pointed out, there is a precedent of sorts at RfC. Finally, I think it is much more against the wiki spirit to propose banning the semi automated Peer Reviews (and yes, I do them at PR so I am biased). How many helpful activities on Wikipedia are banned? Ruhrfisch ><>°° 15:06, 24 October 2007 (UTC) PS Awadewit, you write great articles!
I understand these points. I, too, have to solicit peer reviews. You have presented two different analogies here: a tax and the golden rule. Which do you think we should follow? Your initial proposal was more of a tax - review 2 or 3 articles and get your article reviewed. Your second article was more like an enforced golden rule - 1 for 1. (Perhaps banning bots is against the wiki-spirit, but I also think that their automated analysis of the articles isn't up to snuff yet. When it is, I am all for the bots. It isn't the bots per se I am against - it is the fact that they do not quite work well enough yet. They do feel impersonal, but perhaps I could get over that if they offered good suggestions!) Oftentimes I want to try proposals such as your 1 for 1 idea and see if they work, but in a wiki-environment, I worry that they might become entrenched and hard to remove. I am asking myself right now: if we try this and it becomes entrenched, what is the likelihood that it will make things worse? what is the likelihood that it will perpetuate drive-by reviewing? is it better to start with smaller reforms and see if those work first or is it better to revolutionize the whole system? are we France in the 1780s? (Thanks for the compliment.) Awadewit | talk 17:30, 24 October 2007 (UTC)
I much prefer voluntary measures (as, I would imagine, do almost all editors here). My concern is more that voluntary means are not producing the number of reviewers needed, and this was the only way I could think of to solve that for everyone. I am fine with trying smaller measures first (and even if we introduced a "wikitax" method, I think PR would be a good place to try it out initially). I have made my views on the use of the semi automated peer reviews (it is a script used by a human, not a bot) above (under A Dissenting view). Thanks, Ruhrfisch ><>°° 17:50, 24 October 2007 (UTC)

Exhaustion

One of the effects (on me, anyway) of going through the entire proposal point by point is exhaustion. We have so many more points to go through on peer review alone. :( Would some sort of real-time chat speed up this process? I feel like we are all going to be burned out before we get to any of the other topics we raised outside of PR. Perhaps this is just me, as I am currently inundated with dissertation chapter-revising and grading and... Awadewit | talk 23:23, 24 October 2007 (UTC)

I'd like to hear other opinions on this. In favour of the detailed approach we're taking:
  • It irons out little disagreements, which aren't easily visible if we go faster. Example: my misunderstanding of Awadewit's support for categories for articles but not reviewers.
  • The goal is consensus and a working team whose members trust each other. I think working through details helps us feel like we're part of a team.
Against:
  • It takes a long time, and we are seeing some decline in participation, so perhaps it is counterproductive.
I like the idea of real-time chat. (Actually I'd really prefer to get us all in a room and talk face to face.) Can a chat be arranged? It would require some coordination, and might omit some participants here (I believe we have UK, US, and Middle East time zones represented here).
Personally I have the patience to keep going like this; I think we're settling in and so long as we're not impatient, and are willing to allow that participants can't necessarily post to the workshop every day, I think we'll be OK. What do others think? Mike Christie (talk) 01:21, 25 October 2007 (UTC)
A real-time chat is a good idea, in theory. But for one, it's an extra technical step that some may not be prepared to make, and may isolate them. Others may simply not want to get into "chat rooms" for whatever reason or another. Plus, with the multiple time zones, agreeing on a single date/time for a meeting will likely be darn near impossible; you'd **never** get **everyone** in that wants to be there. Dr. Cash 01:47, 25 October 2007 (UTC)
  • I just thought it might speed up the process. Although I am in the US, I can pretty much adjust myself to any time zone. I have a flexible schedule. I thought it was an idea worth raising. Face to face would be a good idea, but I think that is off the table. :) Awadewit | talk 03:50, 25 October 2007 (UTC)
What would be the advantage of a real time chat? I don't see that it's a good idea at all. Would I be more likely to change my mind about what seems to me to be bleedin' obvious in a chat room than I would be on here? No. --Malleus Fatuarum 03:11, 25 October 2007 (UTC)

Comments from SandyGeorgia

I've been exchanging notes with Sandy while she's been posting here; you can read our exchanges on our talk pages if you are interested. I think Sandy has raised some points we haven't considered; conversely, I think the group has good reasons for some of the suggestions she has raised objections to.

It strikes me that we have an obligation to engage with interested editors who come by with opinions about the work in process here, but we can't always restart from scratch when someone shows up. What I propose to do to address this is have a conversation with Sandy on our talk pages which will try to condense her concerns to a short thread, and perhaps resolve some of them. Once we're down to some condensed version, I'll raise a single thread here to discuss them. That should reduce the impact on ongoing discussions, while giving appropriate effort to integrating a new participant. I also think that as facilitator it's my job to worry about the integration of new participants in the workshop.

Any thoughts on whether this is the right way to go? Mike Christie (talk) 03:22, 25 October 2007 (UTC)

It works for me; I'm sorry I came so late to the process and "made more work". I'd also like to hear from Gimmetrow and avoid losing pieces of articlehistory to miscellaneous talk page archives, but think we should gel this down off of this page, so as not to slow you all down. SandyGeorgia (Talk) 03:25, 25 October 2007 (UTC)

I think its definitely the right way to go. --Malleus Fatuarum 03:31, 25 October 2007 (UTC)

Some thoughts

Because most articles only receive one or two reviews at PR, I think that it is to their benefit to go through GA, PR, and FA. The more people who read an article and who comment on it, the theory goes, the more mistakes will be caught and the better the article will become (this rests on the assumption that the reviewers know what they are doing, of course). I think that a well-organized GAC, PR, and FAC are all good ideas. A rigorous and continuous review system slows down the drive to FA, catching much that would have been missed and informing editors of large problems before they reach FAC. GAC seems to be working better now, although there are not enough reviewers. PR needs an organizational structure in my opinion. I've been sure why it isn't divided by subject matter. I feel completely unqualified to review certain topics while I have to scroll through a list of 100 articles to find those that I do feel qualified to review.

One of the most frustrating experiences for me at FAC is MOS-compliance. I cannot keep up with its changes. Once, months ago, I proposed a sort of "MOS-prep-room" for articles going to FAC. Articles that wanted to be nominated for FAC (and were not obvious WP:SNOW candidates) could go through a process of MOS polishing by those who are experts in it. That way, the only discussions at FAC would be related to prose and substance, and not dashes, dates, etc. Another easy way to solve this problem is to choose a common book of style, such as Chicago. Then there would be no more debate. That seems like a dim possibility, though.

  • Dividing the articles by topic might encourage more people to do peer reviews.
  • Providing some guidance on how to do a good peer review might help new editors.
  • Separating MOS compliance from FAC would encourage good content editors to submit more articles. Awadewit | talk 02:22, 13 October 2007 (UTC)
It'd also be helpful (and save 75% of time) if reviewers identifying MOS issues addressed them at the time rather than spend an equivalent amount of time pointing them out (unless there are loads I guess..)cheers, Casliber (talk · contribs) 10:58, 13 October 2007 (UTC)
If by "addressed them" you mean fix them, I agree 100%. Whenever I read the discussion of a candidate, & find someone insisting that the wrong dash is being used (or any other trivial point from the MoS), I get a little angry; it's hard not to take this as anything other than the commenter has nothing useful to say about this article -- yet is so compelled to say something that they have no shame over saying something silly. -- llywrch 16:40, 15 October 2007 (UTC)
This is an ongoing debate, and I see little chance of it ending. MoS concerns are legitimate, and reviewers can review quite a few articles if they are not expected to make changes themselves to each article they review. I'd rather have several articles reviewed by a competent MoS reviewer than to have one copy edited and MoS-fixed by an editor while nine others go unreviewed. — Brian (talk) 07:20, 16 October 2007 (UTC)
It is one thing to point out that someone has not taken the time to format their references or dates consistently (tiring as this is) but it is another thing entirely to type out "you have a 'the the' in the fourth paragraph of the second section". Couldn't the editor just have removed the extra "the"? Such comments make FAC feel hostile rather than cooperative, in my opinion. Awadewit | talk 07:33, 16 October 2007 (UTC)
  • (undentage) Er, um. well. Little issues and big issues. Easy to get caught up in the former, because they can be so irritating. :-) But I really wanna... I dream... I have a dream.. I have a dream that one day little FAC'ers and GAR'ers will hold hands and play in the streets.. secure in the knowledge that the missions of FA and GA and PR are ever-so-crystal-clearly-spelled out, and have no overlap. I wanna see a MoSPitStop (is that GA?) and a ContentCheckupcenter (is that the A-rating system of the various wikiprojects?). I wanna... uh.. I'm drooling, sorry...
  • Oh I wanna have more proactive PR and GA and FA. Remember {{sofixit}} --Ling.Nut 07:41, 16 October 2007 (UTC)
  • Perhaps some of the folks suggesting reviewers do MOS fixes themselves aren't really aware of how utterly tedious and time consuming it is; unless I'm busy IRL, I spend hours daily doing MOS fixes. I once spent a full 24 hours working on a bird article. I've spent days doing MOS fixes to FAC and FAR articles to the point that I am the top editor on many articles I've never made a content addition to. It's not fast, it's not easy, it's not fun. I also disagree that it's hard to keep up with MOS; most of the core MOS concerns don't change often, and many of them are now fixable with scripts. GAC doesn't review for MOS, some editors are objecting to the automated reviews at PR, PR is backlogged, so unfortunately, MOS issues don't get picked up until FAC. Where else are they going to be detected if GAC doesn't do it? SandyGeorgia (Talk) 01:18, 25 October 2007 (UTC)
Although full MoS compliance isn't one of the GA criteria, when conducting a GA assessment I certainly look for obvious MoS issues. I can't pretend to catch or even know them all (a disadvantage of a single-reviewer system), but IMO ideally there should be no significant MoS problems in any article that passes GA. Perhaps this function should officially devolve to GA, as it's a time-consuming process that GA—with wider participation than FA—may be able to handle better. EyeSereneTALK 11:55, 25 October 2007 (UTC)
EyeSerene, your comment got me thinking; I'm going to type up a longer analysis/proposal later today as I have time. I suspect the PR and the FAC backlogs and issues could be solved if, as you say, there were "no siginificant MOS problems in any article that passes GA". As of now, that's not the situation, and it causes a lot of disappointment to editors who show up at FAC thinking their article is almost there. The sooner editors can learn Wiki's inhouse Manual of Style, referencing requirments, etc., the more effective *all* of their edits to all articles they work on will be. If articles coming to FAC were prepared for FAC, editors might stop using FAC as a proxy for PR and FAC can get back to serious content review for prose, neutrality and comprehensiveness. The problem now is that FAC has become a dumping ground for unprepared articles as PR is backlogged and GA doesn't necessarily encompass a complete review for MOS or reliable sources (the biggest shortcoming of all review processes IMO). SandyGeorgia (Talk) 14:58, 25 October 2007 (UTC)
I think it's definitely worth considering. I've often thought that some sort of MoS-based GA reviewer checklist would be useful, and would educate us reviewers as well as those editors whose articles we review (and since the GA process invites reviews from anyone, would be essential to new or inexperienced reviewers anyway). It would mean expanding the GA criteria slightly, but even if only the most common MoS issues seen at FA were addressed it should help. Incidentally, since FA reviewers are perhaps more MoS-aware than those at GA, and such a proposal would benefit from their input, closer cooperation may help to clear out some of the misconceptions between the two processes. EyeSereneTALK 19:19, 25 October 2007 (UTC)
You are thinking exactly the way I'm thinking. Or I should say, you influenced me to think that way :-) I'm working on it, but to be honest ... ahem ... Awadewit proposed this idea a long time ago, but I resisted because I know I'll get blasted by critics as soon as I put something to paper. This idea that there is some schism between GA and FA just has to end; some of my closest Wiki friends are GA folk. And, I can so easily put together a list of the most common deficiencies that show up at FAC, but as soon as I do that, I'll get blasted as someone who overly focuses on certain aspects. Ah, heck. All you can do is try :-) Just so no one is caught unawares, I'm putting together ideas that would move GA in the direction of picking up extra work in terms of more comprehensive reviews of a couple of items, and extra credibility and recognition hopefully to go along with it :-) SandyGeorgia (Talk) 19:33, 25 October 2007 (UTC)
I also think it would be helpful to clearly indicate what the next step for the article after PR is (within the Peer Review request). I think there would be three possible next steps: FAC, GAC, or general improvement of the article. If this were clearly shown (and some editors do indicate this already), then I would give priority to FAC, followed by GAC, and then just improvement. I also think the level of detail in the review would be highest for FAC, medium for GAC, and "Get some references and fix your spelling" might work for some improvement requests. If we go to a new system, I would hope it would indicate the next step after PR clearly. Ruhrfisch ><>°° 20:00, 25 October 2007 (UTC)
I'm not sure the process of review can be standardized; it really depends on the article and the editor. Many experienced FA writers, for example, go straight to FA without even a peer review and without GA. Some get their peer review extra-officially, by approaching editors they know. In one case, even though an article I was helping to prepare for FAC got a solid peer review and was ready for FAC and really had no reason to get a GA rubber stamp in between, I recommended to the main author that he get that GA stamp to show broad consensus existed for the article, as it is subjected to disruptive and POV editing. So, I'm not sure there's a natural or logical progression that all articles or editors must follow. I'm focusing on what we're missing; getting some fundamental issues in place before unprepared articles show up at FAC, resulting in shock and disappointment to the nominators, so that FAC can better focus on in-depth content review. I understand clearly the purpose of peer review; it's for whatever the editor needs at the time PR is approached. I understand the purpose of FAC; it's to assure that articles represent our best work, with no known omissions. I think we can better carve out a more clear niche for GA, since it's no longer viewed as only for short, good articles. SandyGeorgia (Talk) 20:13, 25 October 2007 (UTC)
  • Just to clarify, I don't want to standardize Peer Review or make it some lockstep process. My thought was more that it would be helpful to always specify what the submitter's next goal for the article is after PR, so that reviewers could do their job better. I have taken 3 articles to FA and always went through PR, but skipped GA on one. Ruhrfisch ><>°° 01:17, 26 October 2007 (UTC)
There was quite a recent discussion about including more MOS issues in the GA criteria here. Epbr123 20:22, 25 October 2007 (UTC)
Thanks for that lead, Epbr; I think there's a more collaborative spirit now, thanks to this effort. SandyGeorgia (Talk) 21:27, 25 October 2007 (UTC)
(edit conflict) I'm still pretty new to the whole review process myself really—certainly compared to some of the editors posting here—and still have much to learn. I'm only aware of this antagonism because I've read various comments in discussions I've stayed out of, but it only seems to be certain editors that have problems. From what I've seen, many others work across all quality processes anyway... and anyone who's served their time has my respect, regardless of where they've served it ;) I'm looking forward to seeing your ideas... EyeSereneTALK 20:31, 25 October 2007 (UTC)
Would it make sense, then, for me to put together a list of the most common shortcomings that show up at FAC (they are not confined to MOS issues) to get a sense of whether GAC would be interested in expanding to cover some of those issues? I think that would get some pressure off of PR and FAC, and PR could go back to being a one-stop for whatever you need while FAC could focus more deeply on content. SandyGeorgia (Talk) 21:12, 25 October 2007 (UTC)

(unindent) I think it's a great idea, although I don't want to preempt any discussion that will arise when we get to looking at GA on Mike's list. IMO it can't do any harm to work on it and maybe put feelers out, but I'd like to know what others think. EyeSereneTALK 21:29, 25 October 2007 (UTC)

I think it would be great too, and, in this collaborative spirit, once the list is established, I'm sure this workshop can find someone to present the idea in such a way that it won't get shot down immediately. Geometry guy 23:06, 25 October 2007 (UTC)
I hope you're right, Gguy, but I'm not holding my breath. I've started putting it together; it will take me some time, and I fully expect to be blasted once I put it out. I'd love to be wrong :-) SandyGeorgia (Talk) 23:23, 25 October 2007 (UTC)
I also think that could be really helpful. I do take your point, in particular about taking some of the pressure off FA. --Malleus Fatuarum 23:21, 25 October 2007 (UTC)

Three more questions for consensus

I've marked the nomination proposal as having reached consensus. Here are two more from the undecided list; two, because both seem fairly uncontroversial. Mike Christie (talk) 02:07, 23 October 2007 (UTC)

  • What categorization should be used?
  • Use the WP 1.0 hierarchy, which is
Arts · Language and literature · Philosophy and religion · Everyday life · Society and social sciences · Geography · History · Applied sciences and technology · Mathematics · Natural sciences
  • Use the FA hierarchy, which is:
Art, architecture and archaeology · Awards, decorations and vexillology · Biology and medicine · Business, economics and finance · Chemistry and mineralogy · Computing · Culture and society · Education · Engineering and technology · Food and drink · Geography and places · Geology, geophysics and meteorology · History · Language and linguistics · Law · Literature and theatre · Mathematics · Media · Music · Philosophy and psychology · Physics and astronomy · Politics and government · Religion, mysticism and mythology · Royalty, nobility and heraldry · Sport and recreation · Transport · Video games · Warfare
  • If you're asking for opinions, then my preference would be to align with the WP 1.0 categorisations. --Malleus Fatuarum 02:14, 23 October 2007 (UTC)
  • I am in favor of WP 1.0 as well. Awadewit | talk 02:18, 23 October 2007 (UTC)
  • Yes, the 10 topics of WP 1.0 seem about right to me, and they are also more widely used. Indeed, it would make sense, I think, to group the finer FA classification (which is only used on WP:FA) under the WP 1.0 headings. Geometry guy 08:16, 23 October 2007 (UTC)
  • Sorry, but the eleven are way too broad. Arts and Natural Sciences? These are omnibus categories. I think History is also too generalized, but otherwise support the FA categories. Marskell 20:48, 23 October 2007 (UTC)
I think breadth is appropriate for PR. I would also give reviewers the opportunity to summarize their interests in free form (see below). Given the level of participation, 10 categories seems about right. We don't want a list with only two or three people in each category! But generally, a peer review of a biology article by a physicist would be great, as would a peer review of an architecture article by an art historian (the latter are not distinguished by the FA classes either, which further mixes these classes with archaeology). Geometry guy 21:16, 23 October 2007 (UTC)
  • I would agree on using the WP1.0 categories as well. They seem a bit more simple and straightforward, and should suffice for a simple categorization like this. Dr. Cash 01:51, 25 October 2007 (UTC)
Hm, actually I do want a category with, say, five or six people. Arts? By itself? That doesn't seem useful or browsable to a nominator looking for a specific person to review. Marskell 21:49, 23 October 2007 (UTC)
  • No categories; if the backlog at PR is reduced, they won't be needed. If categories must be used, then FA categories; WP 1.0 is too fragmented, too many. SandyGeorgia (Talk) 03:05, 26 October 2007 (UTC)
Could you explain in what sense 28 categories (FA) is less fragmented than 10 (WP 1.0)? Or perhaps you have not understood the proposal. We are only talking about the top level here. Geometry guy 07:07, 26 October 2007 (UTC)
  • How does a peer reviewer add themselves to the list of available reviewers?
  • Add a template to their User page or User talk page.
  • Add their name, possibly using a template, to some location on the PR page.

I don't really have a view on how reviewers add themselves to a list of reviewers, mainly because I believe that anyone ought to be able to review any article, whether they're on a list or not. --Malleus Fatuarum 02:21, 23 October 2007 (UTC)

I hadn't thought of that. I would have thought these options do not prevent anyone from reviewing anything they like. If anyone disagrees, please say so, otherwise I'll incorporate a note to that effect on the front page. Mike Christie (talk) 02:33, 23 October 2007 (UTC)
I don't think anyone has said we should restrict reviewers from reviewing whatever they like. I thought we had made that clear before. Is this a list of categorized reviewers? I am really only in favor of that - a gigantic list of hundreds of reviewers is not helpful, IMO. I thought the whole point of the list was to find specialized reviewers when one wanted them. Awadewit | talk 02:37, 23 October 2007 (UTC)
I'm not generally favour of that idea either. For one thing it would be an m:n relationship, but more importantly because it would encourage "incestuous" reviews. --Malleus Fatuarum 02:47, 23 October 2007 (UTC)
Well, we shouldn't get bogged down in this again; I will make one last statement. I am in favor of having both "general" and "specialized" reviews. The only way to ensure that all topics are covered as well as wikipedia can cover them at PR is to have reviewers list their interests - that way an editor can solicit either a general or a specialized review. After 15 FAs and FLs and numerous other articles, I can unequivocally say that I have been helped by both specialists and generalists (insert long list of editors here - see my userpage). I would not like to restrict myself to either kind - it would be foolish. Awadewit | talk 02:59, 23 October 2007 (UTC)
  • What I can unequivocally say is that listing an interest is not the same as having an expertise. I thought that lesson had been learned. --Malleus Fatuarum 03:06, 23 October 2007 (UTC)
  • Let's see what others think on this matter rather than bickering between ourselves, shall we? Awadewit | talk 03:13, 23 October 2007 (UTC)
  • I have been made to feel like an interloper in this discussion on more than than one occasion now; and probably I am, as I simply stumbled across it. But I have taken offence at Awadewit's accusation of bickering, so it's now time for me to step away from this project. Good luck with it, I hope you manage to make a difference. --Malleus Fatuarum 04:45, 23 October 2007 (UTC)
I had left out the question of a reviewer listing their categories of interest in the hope that it was separable from the question of how a reviewer should list themselves. The reviewer-in-categories question seems to have several possible answers, as the article nomination method did. I'll start a separate indent thread below to try to give those possibilities for comment. In the meantime, could we use this thread to comment on whether there is any preference for how a reviewer indicates their interest in being a peer reviewer -- on PR or on the reviewer's talk page? Mike Christie (talk) 03:15, 23 October 2007 (UTC)

(outdent) The two questions are related though. I absolutely agree that anyone should feel free to peer-review articles; community contribution goes to the heart of WP itself, and I don't believe anyone has disagreed with this. The contention seems to be whether or not there should even be a list of reviewers at all. Personally, along with Awadewit, I see the value of such a list arising from the fact that an editor can make a specific request for an 'expert' review (although this overlaps to some extent with the current A-class processes), but we'd need to be careful that this does not act to exclude casual reviewers. Malleus makes a good point in that any list would need to handle reviewers who have listed interests in multiple categories - in which case user-page template(s) may not be ideal. Manual listing on a PR page might be more straightforward. EyeSereneTALK 09:34, 23 October 2007 (UTC)

I agree absolutely that anyone can peer review, listed or not. The benefit of listing is twofold: to identify particularly active peer reviewers, and to identify their interests.
The User page or User talk page template approach could easily handle multiple interests using a format such as {{peer-reviewer|interest1|interest2|...}}: again the template would work using categories, and the bot would read the categories to produce the list. I think there is still a case for doing it this way, but the case is not as strong as it was for the peer review requests. Geometry guy 19:47, 23 October 2007 (UTC)
  • I do not and will not add userboxes or templates to my userpage, but I would sign up on the PR page. SandyGeorgia (Talk) 03:12, 26 October 2007 (UTC)
The templates could be invisible. Does that change things for you? Geometry guy 07:07, 26 October 2007 (UTC)

A third issue, with several possibilities (and perhaps more that I've missed). Please comment or indicate support or opposition below. Mike Christie (talk) 03:20, 23 October 2007 (UTC)

  • Should editors who list themselves as peer reviewers list some category in which they are interested?
  • They should not have the option of listing categories, but should only be able to list their names
  • Not sure of the benefit to anyone here. Awadewit | talk 22:05, 23 October 2007 (UTC)
  • They should be able to list a category that is one of the agreed category titles. No other categories would be permitted.
  • They should be able to list their interest in any category, and create new ones if necessary.
  • Sounds like a potential nightmare with proliferating categories everywhere. Awadewit | talk 22:05, 23 October 2007 (UTC)
  • They should be able to list their interests in free form text, in addition to the category information, if any
  • The should be required to list themselves in a category; uncategorized reviewers would not be listed
  • I don't think this is a good idea - requiring people to list their interests goes against the idea of having both general and specialized reviews at PR. Awadewit | talk 22:05, 23 October 2007 (UTC)
  • They should be able to list themselves in more than one category.
  • If we have categories - obviously. Awadewit | talk 22:05, 23 October 2007 (UTC)

I think the case for maximum flexibility here probably trumps most other considerations (although I'd be happy to be proven wrong!): reviewers can provide one or more categories, but don't have to; they can also provide free-form text, but don't have to. I'm against reviewer-created categories, though, as it complicates the system too much. Geometry guy 19:51, 23 October 2007 (UTC)

Yes, I think we should stick to the (going by consensus so far) WP 1.0 categories. Additional free-form text can clarify specialisms within a category. Taking Gguy's example above I'd guess something like {{peer-reviewer|interest1|comment1|interest2|comment2...}}?
If we do go with templates, it's possible not all reviewers will want these on their user pages. I suppose they could go on any sub-page, so they could be hidden away somewhere, or reviewers can manually add the relevant cats, but I still think there should be some way for reviewers to add themselves to a list directly. EyeSereneTALK 20:14, 23 October 2007 (UTC)
Using a template like that will restrict them to the eleven broad categories, which makes little sense. I'm not sure that we need templates, though I suppose they'll keep things tidy. Marskell 21:03, 23 October 2007 (UTC)
The templates would not need to be visible on the User/User talk page, or they could be optionally visible: that is easy to code. I wonder how many listed reviewers are there likely to be? 100 perhaps? Among 10 categories, maybe they have 2 interests on average, so 20 reviewers per category. There would also be free form text listed by each reviewer, if they want. I think searching through 20 people for the best matches is quite a good system. But I'd be interested to hear Awadawit's view. Geometry guy 21:27, 23 October 2007 (UTC)
I like free-form lists like this because then reviewers can easily list several categories. I could easily see myself writing: "Eighteenth- and nineteenth-century European literature and history; children's literature; lay reader for science articles" or some such entry. However, if we have hundreds of reviewers, it would not be possible to look through such a list - I think 100 is about the max. Even that is tedious. Awadewit | talk 22:05, 23 October 2007 (UTC)
I agree. So I think breaking down the list into groups of around 10-40 reviewers with listed free-form interests would work quite well. Geometry guy 22:16, 23 October 2007 (UTC)
Free form lists, and I don't agree with WP 1.0 categories or any categories. I participate in PR because it's a fun and easy way to add to the Project; I do not like the idea of turning it into a top-heavy, instruction-laden, or overly formal process, and won't participate at all if the page gets as complex as the GA pages are. I like looking through the PR list for a random topic that interests me. SandyGeorgia (Talk) 03:12, 26 October 2007 (UTC)
You refuse to participate if it is not exactly the way you want? That seems a little silly, don't you think? Since wikipedia works by consensus, I don't think anyone gets exactly what they want. Besides, then you will just have to fix all the dashes at FAC, anyway. :) Awadewit | talk 03:19, 26 October 2007 (UTC)

(outdent) In which case, are categories even necessary? There seem to me two main options for the list page itself:

  • Reviewers list themselves in arbitrary order, along with a free-form comment stating any preferences
Comments: per Awadweit, reviewers can list several categories/interests; possible scalability problems if the list gets large (eg hard to search); why bother using categories at all when more detail can be given in free-form text?
  • Reviewers list themselves by category, perhaps with a 'Misc' or 'Unspecified' section for those that don't want to specialise
Comments: PR user template categories can be used to organise the list; scales better (eg easier to search); reviewers with multiple interests will be listed in multiple categories

From an organistational POV, my preference would be for the second option, but there are other options (eg a tabular list). Maybe others have better ideas? EyeSereneTALK 17:03, 24 October 2007 (UTC)

Summary of the discussion so far

Here's where I think we are in this discussion, with a suggested synthesis.

Update: I have split the categorization question in two based on Awadewit's comments. Please revert if I've mischaracterized. Mike Christie (talk) 21:39, 24 October 2007 (UTC)
  • What categorization should be used? See below Malleus Fatuarum, Awadewit, and Geometry Guy have supported the 1.0 categories. Marskell preferred the FA categories, partly because of the need to have detailed information about what a reviewer is interested in.
  • I did not support the WP 1.0 categories for reviewers, only for peer review requests for articles. I am in favor of "free-form" text for the list of peer reviewers, as my example of how I would describe myself was supposed to demonstrate: "Eighteenth- and nineteenth-century literature and European history and literature; children's literature; lay reader of science articles". However, I do see the benefits of categories if the list becomes too long - over 100. Awadewit | talk 17:22, 24 October 2007 (UTC)
One of those funny text-communication issues: I interpreted your earlier comments to mean that you didn't see the point of reviewer lists unless reviewers are listed by expertise/category! Anyway, I'm also in favour of free-form descriptions, but I think we should be aiming for a longish list, hence subdividing such lists by categories would be a useful search aid. Geometry guy 18:30, 24 October 2007 (UTC)
  • What categorization should be used for the article listings? (Added after comments above.)
I don't know if Marskell's comments would apply here, because his objection related to the reviewer categorization. Other than Marskell I have seen no objections to the use of WP 1.0 for this. Mike Christie (talk) 21:39, 24 October 2007 (UTC)
  • What categorization, if any, should be used for listing reviewers? (Added after comments above.)
Here I saw two concerns: Marskell felt the WP 1.0 categories were too broad to be used for this purpose; Awadewit preferred not to have any categorization of the reviewer listings, but allow reviewers to express their interests with a text field. Mike Christie (talk) 21:39, 24 October 2007 (UTC)
So the obvious comprimise would be to have interests expressed in a text field, but use the 1.0 categories to make searching for a reviewer easier. Geometry guy 21:47, 24 October 2007 (UTC)
  • How does a peer reviewer add themselves to the list of available reviewers? There were no strong opinions on whether a peer reviewer should add something to their user page or the peer review page; questions about ease of use and ability to get their listing in the right category were seen as more important.
  • Should editors who list themselves as peer reviewers list some category in which they are interested? Geometry Guy summarized this as "reviewers can provide one or more categories, but don't have to; they can also provide free-form text, but don't have to". I think this is pretty accurate; I didn't see anyone disagreeing with this approach.
  • Without a peer reviewer listing some category of interest, I don't really see the point in having the list, frankly. What would be the point? Please explain. Awadewit | talk 17:22, 24 October 2007 (UTC)
Ideally, all reviewers would list categories and free-form interests, but it is also useful to know which editors are actively involved in PR: for many articles, the nominator may just want to find an active general reviewer. Geometry guy 18:30, 24 October 2007 (UTC)
But if someone posts a request for peer review, isn't that pretty much soliciting a general review? Are we saying the editors have to individual peer reviewers every time they post an article? That would be a terrible idea, in my opinion. Awadewit | talk 20:33, 24 October 2007 (UTC)
I agree, it would be a terrible idea. Rules are generally bad, but information is generally good. For example, a nominator might recognise a name on the list of reviewers, and post a message on their talk page. Geometry guy 20:55, 24 October 2007 (UTC)
Yes; to clarify, I think what we've agreed so far is that there should be two lists: articles requesting reviews, and reviewers willing to do reviews. So there would be two methods of getting a review -- you can ask someone from the reviewer list, or post the article to the article list and wait. Similarly there are two ways of reviewing -- pick something that is waiting for a review, or wait for a reviewer to contact you. I would assume that there's no obligation to post your article to the list if you're just going to contact reviewers; and there's no obligation to post your name to the reviewer list if you're just going to pick things on the article list to review and don't want to be asked directly on your talk page. Mike Christie (talk) 21:54, 24 October 2007 (UTC)
No to one point! Your assumption is wrong (i.e., has not been expressed anywhere on this page). All articles for peer review must be added to the PR list (but this is done simply by putting a template on the talk page). Contacting reviewers is then an additional optional step. A private arrangement between individual editors can happen, but does not need a process. The point of PR is to get as much input into each article as possible. For this, the article must be listed. Geometry guy 22:20, 24 October 2007 (UTC)
Well, I don't have a strong opinion either way. I think it's realistic to acknowledge that if a reviewer lists their name and interests on the PR page, someone may just contact them directly without submitting their article to PR. That's not in the scope of our process, true: our scope is increasing input to "articles that are submitted to PR". But it'll probably happen. Or do you think it shouldn't even be mentioned in our notes, because it relates to something that is not PR? Mike Christie (talk) 22:31, 24 October 2007 (UTC)
Actually, on reflection, I think you have a point. PR can also be a dating service, and once the date has been made, the agent does not attend! I have expressed a similar view about GA in the past, but it might fit PR better. I'd be interested to hear other editor's views on this. Geometry guy 22:36, 24 October 2007 (UTC)
  • I think that your idea of a "dating service" fits perfectly with the idea that I've stated several times already, that PR ought not to be seen as a process within any kind of a hierarchy. So I'd be quite in favour of it. --Malleus Fatuarum 23:15, 24 October 2007 (UTC)

Marskell, if a reviewer can use the "free form text" comment to make it clear what subset of a category they specialize in, do you still object to the use of the WP 1.0 categories? I.e. if I list myself under "History", but add text saying "Only Anglo-Saxon kings and battles", then I would be listed with that comment in the history section. What do you think of that approach? Mike Christie (talk) 12:52, 24 October 2007 (UTC)

Archives

SandyGeorgia has made a very good point about maintaining a record of peer reviews (and in fact this point is even more important for GA reviews, because editors need to be able to find a GA review easily to verify that a GA has been passed properly). The problem is that talk pages get archived, and so links get broken. However, it seems to me that there is a similar problem with the current peer review process: if an article receives a second peer review, then the first one gets moved to an archive. (Possibly there is a bot fix for this: I haven't investigated in detail what happens for multiple PRs.)

So, what does the record need to provide? Well, when a new content review starts (be it GAN, FAC, PR, GAR, FAR...) a new reviewer would like to be able find previous reviews of the article and read them. So it seems to me that the logical thing to do is to store all reviews of an article in one place. The obvious place would be a subpage of the article talk page such as Talk:ArticleName/Reviews. This could either be done by conducting the review on such a page, or by appending the review to that page once it is complete. The links to reviews from e.g. ArticleHistory would then be static. Geometry guy 18:12, 25 October 2007 (UTC)

When GimmeBot runs through, Gimmetrow fixes any outdated archives on peer reviews. I manually do any I find when I'm correcting articlehistory errors. Editors should have followed the instructions, but they don't always. At any rate, all events except GA events (peer review, FACs and FARs) are established on separate pages, that can be archived to articlehistory. GA is the only different process, and IMO, that's a problem that causes of lot of excess work. Fixing a FAC or PR archive is easy; finding and fixing GA errors is timeconsuming. One of the issues that concerns me in the proposal above is that, rather than GA moving towards other established processes (articlehistory, PR, FAC, FAR, and all other featured processes), the proposal is to move away, replicating the weakness in the GA process and moving PR to talk pages. SandyGeorgia (Talk) 18:33, 25 October 2007 (UTC)
PS, I disagree that the obvious place is talk page subpage. It would be fantabulous if GA would move towards other established processes. All featured article processes and peer reviews follow standardized naming conventions, and are not subpages of talk pages. SandyGeorgia (Talk) 18:34, 25 October 2007 (UTC)
SandyGeorgia, could you elaborate on "rather than GA moving towards other established processes...the proposal is to move away, replicating the weaknesses in the GA process". We haven't proposed anything yet for GA, as far as I know, so I assume you meant PR, where we have come to a consensus of sorts, but what weaknesses in the GA process are you referring to? Awadewit | talk 21:49, 25 October 2007 (UTC)
All of the featured processes and peer review have separate pages for those reviews; GA is the only process that uses talk page entries which get subsequently buried in talk page archives. The proposal has been to move PR to talk pages, which replicates a weakness in GA. It becomes impossible to find GA events in talk page archives as a page grows and they aren't registered on individual pages as discrete events in the articlehistory. This lowers credibility and accountability for the GA process. It would be a serious mistake, IMO, to have PR move in that direction. SandyGeorgia (Talk) 22:36, 25 October 2007 (UTC)
Could you explain how having the GA review on the article's talk page lowers the review's accountability? I can't really see that. Even if we haven't solved the practical problem of linking to it yet, I don't see that as an insurmountable problem. Surely someone must possess that computer knowledge. I live with a computer nerd and he always tells me "all things are possible". Awadewit | talk 00:35, 26 October 2007 (UTC)
Anyone can pass GA. When you add to that the fact that you can't always find who, how, when or why a given article was passed, that affects credibility, or at least perception of a process that lacks accountability. SandyGeorgia (Talk) 00:44, 26 October 2007 (UTC)
I am not in favor of designing proposals around templates such as the "article history" template (which I spent an hour one night trying to understand and then simply gave up on). The proposals should further article content. The templates can always be changed. Those templates are not the most important part of wikipedia, in my opinion. Awadewit | talk 21:49, 25 October 2007 (UTC)
This is one of those approaches that I find hard to understand <smile>. Articlehistory is installed on thousands of articles, which include *all* of Wiki's best (featured) articles. It was broadly discussed and endorsed (including by Raul) as a means of decreasing talk page clutter. It's not likely to change; adapting to it rather than the other way around would seem to make sense. It's really not that hard to use if you install Dr pda's prose size script, and if you don't understand or want to deal with articlehistory, all you have to do is use the old templates and wait for someone who does speak articlehistory to install it. If you're interested and if you explain where you're hung up, I'll help, but it's not something you have to do (if you notice, Gimmetrow and I run through all FAs and take care of them). At any rate, regardless of whether any event in the article's history uses the {{articlehistory}} template, we still have the problem that no templates point to GA events, as they are recorded in talk page archives rather than separate pages, and finding a GA event often requires a hunt. SandyGeorgia (Talk) 22:40, 25 October 2007 (UTC)
"No templates point to GA events"? There are loads of ArticleHistory templates pointing to GAR archives, and to talk pages and/or talk page archives of GA reviews. I accept your point and have suggested a solution: please don't exaggerate the problem. Geometry guy 22:58, 25 October 2007 (UTC)
Awadewit asked a question; I'm trying to answer it. Featured article events and peer reviews have separate pages; currently GA events do not. They are buried in talk page archives; this makes them hard to find. It's curious to me why there is always GA resistance to standardizing events to agree with other estalished procedures. But it's not a big deal to me. Just trying to have the conversation. SandyGeorgia (Talk) 23:18, 25 October 2007 (UTC)
(I have ceased using "article history" templates after I messed up coding them. Any assistance you could provide, SandyGeorgia, would be much appreciated; perhaps you could leave a little tutorial on my userpage.) However, I still feel that we should design the best PR, GAC, and FAC possible first and then worry about the templates. If they have to be changed, I feel that such a thing is do-able. I don't want to feel constrained by a template. I don't want to feel that we can't change the way FAC is run to improve the way sources are reviewed, for example (I share your concern), because of a template issue. I feel that article content should be our first priority and article tagging (is that the proper term?) our second or third concern. Awadewit | talk 00:35, 26 October 2007 (UTC)
Pick an article that you want to update, and drop me a line so I can walk you through it. But honestly, if you don't have Dr pda's articlehistory script (which automates a lot of it), it's better/easier to just leave the work to someone else who does have the script. SandyGeorgia (Talk) 00:44, 26 October 2007 (UTC)
I think SandyGeorgia is saying that the current proposals move PR in the direction of GA, whereas in fact we should focus on moving GA in the direction of PR and FA. The main GA weakness is the lack of a review archive, but my suggestion rectifies this, so I'm not sure why this issue needs to be brought up again in this section. Geometry guy 22:52, 25 October 2007 (UTC)
PS. Even though I love templates, I strongly agree with your point that templates should be designed around proposals, not vice versa. ArticleHistory is indeed very complicated. I have to recheck the instructions every time I use it. It is very well coded though :)

If reviews were on a fixed subpage of the talk page then Gimmetrow, GimmeBot and SandyGeorgia would not need to go around fix those links. Also, any reviewer reading one previous review would easily find the rest. Instead of clicking on multiple ArticleHistory links, it is all there in one place. So why is this not the obvious place? To misquote Marskell: "if you had to scrap all of the instructions and write them again, how would you do it?" Geometry guy 22:52, 25 October 2007 (UTC)

From the perspective of a database designer, I can't see why the "permanent link" ought not to be precisely that, permanent. I can't see a single reason good why archiving a review ought to change its primary key. --Malleus Fatuarum 23:05, 25 October 2007 (UTC)
I like to leave the details to the programmers and not get bogged down by that level of detail when we're only tossing out ideas anyway. Where we park a review is secondary to whether it should be parked in a place other than the talk page archives. SandyGeorgia (Talk) 23:20, 25 October 2007 (UTC)
The point I was making is that it doesn't technically matter where it's "parked"; it's not a programming issue, and no reason to get bogged down over it. Just use the real key and the problem disappears. --Malleus Fatuarum 02:00, 26 October 2007 (UTC)
GimmeBot may disagree; it updates old archives according to a standard naming convention, and I'm not sure if a change would mean unnecessary programming. In general, I just don't think it's good practice to change long-standing things often (this has been an issue with the GA process) or to change things that aren't broken, and the naming conventions for peer reviews are long-standing. I guess the thought is that moving peer reviews to talk pages will encourage participation, but I don't see that at all. I asked Gimmetrow to weigh in here from the bot programming side, but I do suspect the final decision may be more about practicality than programmability. SandyGeorgia (Talk) 02:05, 26 October 2007 (UTC)

Key points from the above

There are several interesting comments in the above, but I think the core question can be summarized as:

  • Should the peer review reside on a single permanent page so that it can have a permanent link to it? Or is it OK for it to be on the main article talk page, where it will eventually be archived away?

Please correct this if I have it wrong.

The argument for a single permanent page is that a permanent link is valuable for people who wish to read the key reviews in the article's history. The ArticleHistory template is designed to permit this. The argument for article talk (which I didn't see recapped, but I believe I recall from earlier discussions) is that article talk is more likely to encourage interactive work on the article.

The solution proposed by Geometry Guy is that we use a subpage of the article talk page. I have an additional question I think might clarify things:

  • If we use a permanent page with a link that will never change, does it matter where it is? Would it matter to a bot that might create it, or to the links from the PR listings page, or to the ArticleHistory template?

If the answer is no, it doesn't matter, then are there any other considerations? A separate page won't appear in article talk without either transclusion or cut and paste (either of which could be done by the bot). So is there some reason it matters where the permanent page is, if we have one? The only reason I can think of is that if articles are moved, their subpages would require moving too. On some PR subpage, they would not.

So did I miss any key points? Are those the questions to be answered? If so, let's see if we can get to consensus on this. Mike Christie (talk) 01:09, 26 October 2007 (UTC)

From a bot point of view, I'm almost certain it doesn't matter what the parking place is named (but I will ask Gimmetrow). From a practical point of view, given that there is already a long-standing naming convention and procedure for archiving peer reviews that works, why change the existing naming conventions (which may create unneeded confusion)? Also, if you change the naming convention now, future bots have to account for two different naming sytems; the old one and the new one. SandyGeorgia (Talk) 01:25, 26 October 2007 (UTC)
Which is more important? Making the bot's life easier or encouraging more participation in the review process? --Malleus Fatuarum 02:04, 26 October 2007 (UTC)
Can someone please tell me (or give a diff) how putting peer reviews on a new page, whatever it is called, is going to encourage more participation? I came late to this discussion, I have tried to read it all, but I just don't follow the logic. As I see it, the problem is too many requests for reviews, and not enough reviewers. How does moving where PR takes place decrease requests or increase the number of reviewers or the number of reviews done per reviewer? I made a suggestionabove which is not perfect, but at least addresses this issue. Ruhrfisch ><>°° 02:42, 26 October 2007 (UTC)
I've reviewed a lot of articles, and I don't see that either. I certainly am *less* inclined to get tangled up in an article talk page, while I am inclined to pop in to the independent PR page. SandyGeorgia (Talk) 02:45, 26 October 2007 (UTC)
In my opinion, as both a frequently peer reviewer and peer reviewer-receiver, it is easier to have the review on the talk page because it encourages participation by the reviewer in the article editing process (something mentioned somewhere above that many of us desired). Moreover, more editors from the article are likely to get involved in the reviewing process (which is usually good). While I agree to an extent with what Sandy is saying about not getting involved in petty disputes, I also agree that it is good for reviewers to have an idea of the history of a page before making suggestions. I always try and read the talk pages first as a way to avoid re-starting old debates. This would be facilitated by having the PR on the talk page. I don't think that the only problem is too many requests and not enough reviewers, by the way. Awadewit | talk 03:06, 26 October 2007 (UTC)
What are editors of an article more likely to look at? The talk page or a separate review page? --Malleus Fatuarum 02:54, 26 October 2007 (UTC)
Regular editors who have an interest in the article will look at either; the advantage of a separate PR page is that it brings in independent editors for a fresh look and new input, and editors who don't want to get tangled up in article talk page politics <raising my hand often on that one>. SandyGeorgia (Talk) 02:59, 26 October 2007 (UTC)
I don't think there can be a definitive answer here. The argument for the article talk page is quite reasonable, as Malleus Fatuarum implies: casual editors will encounter a peer review written on the article talk page without the need to click on a link. The argument for a permanent link is also reasonable; anyone who understands the ArticleHistory template would look at a peer review for an article they were interested in. Article talk page peer reviews will eventually scroll to archives; separate peer review pages are never immediately visible on article talk. So there are pros and cons on both sides. Does the ability to link directly to a permanent page in ArticleHistory make a bg difference? It's been argued that it does. Is it useful or necessary to have that peer review linked? Mike Christie (talk) 03:09, 26 October 2007 (UTC)

Crazy, compromise idea taking the best from both sides, and thus everyone will instantly hate

The main problem people seem to have with the article talk page option is the loss of the review when and if the talk page is archived. Could there be a possibility of segregating or otherwise containing the review (maybe some differently colored box) to indicate a closed PR, (like when a closed move proposal is done) that would also somehow cue an archive bot NOT to archive it? That would seem to solve the problem, no? --Jayron32|talk|contribs 03:17, 26 October 2007 (UTC)

No, I have three main problems: that one, don't fix something that isn't broke (PR works, it's overburdened), and I do PR because it's fun. I am not interested in getting tangled up in article talk page politics, and just won't do it anymore if it goes that direction. The advantage of PR is that it's independent and brings in fresh eyes and people who don't want to get tangled up on talk pages. SandyGeorgia (Talk) 03:21, 26 October 2007 (UTC)
I think in order to make progress we have to narrow our focus in individual conversations. I think in this case, the two additional problems you raise need to be addressed independently: that PR isn't broken, it's just overburdened, and that you don't think it should be on an article talk page. I think the question of whether PR is broken or not needs to be a separate thread, and the question of not getting involved in an article talk page may not arise with some of the solutions being considered here.
Sandy, you've commented that changing from PR subpages to an article talk subpage would perhaps make things difficult for the bots, and you've also questioned the need for change when the existing PR page is an established convention. Those are reasonable questions to ask. Can you tell me if you would no longer contribute to peer review if peer reviews took place on a separate page, but it was a subpage of the article? Mike Christie (talk) 03:35, 26 October 2007 (UTC)
Article space is article space, even if a subpage; I like the independence of PR and other review processas as "another place" where independent reviewers contribute because they aren't obligated. The reason PR is fun is that I can give as much as I have time to give, and not have to feel pressured or obligated as I do at FAC or FAR (where feedback is required), and I can maintain my independence from the article politics and personalities. SandyGeorgia (Talk) 03:40, 26 October 2007 (UTC)
OK, thanks for clarifying that. There was a previous strong consensus that article talk was a good thing, but I think it is worth asking the question again given your input. I will create a separate section to ask that. Mike Christie (talk) 03:51, 26 October 2007 (UTC)

I'm curious about how you all perceive these proposals to be received once they leave the relatively protected confines of this page, which few editors are even aware of; why do editors here think the broader community will agree to peer review being moved to talk pages anyway? I doubt the notion of moving PR to talk pages will fly outside of this group, so I'm wondering what the utility is of moving so far forward on the implementation aspects of these proposals without broader community input? I can not imagine a reason that the broader community would agree with moving PR towards GA practices of entries on talk pages, which will simply discourage broader PR input, when the whole point of PR is broad input by peers uninvolved with the article. Confused, scratching my head, just asking, because it seems that there is a notion that consensus among this small group will translate to consensus in the broader Wiki community, and I'm just not seeing it, SandyGeorgia (Talk) 17:55, 26 October 2007 (UTC)

The question of when to take this conversation outside this group has come up, and I think nobody is under the illusion that whatever we propose is going to be agreed to instantly. The intention is to come up with a coherent proposal which can be suggested in the appropriate forum; at that point the whole discussion will presumably start over for everyone who was not involved here. This workshop is an idea factory, not a decision-making body. If we come up with good ideas, they'll gain support when we eventually propose them. If not, they'll be shot down.
We're also consciously not advertising this page, because one of the goals is to have a small, coherent group that includes representatives of all the different WP review processes, and make that group into an effective team. A key problem has been acrimony in discussions between the different groups; we'd like to improve that too. Of course anyone who finds the page is welcome to join in, but because a key goal is collaboration, and because these topics generate a lot of discussion even in a small group, I think the best outcome is a smaller group of people who are committed to working in a team.
Does that clarify things? Mike Christie (talk) 18:04, 26 October 2007 (UTC)
Yes, it does, and it's in line with what I thought was the idea here; thanks Mike. So my question is, why is the group focusing so early on a level of detail of implementation of the ideas (as in what page discussions will reside on) when broader input as the substance of the proposals will be needed anyway? It just seems that a lot of time could be chewed up in the nitty gritty of implementation when the proposals could be rejected by the broader community. I guess I'm urging to stay focused on ideas and away from specifics of implementation, like which page discussions should reside on. SandyGeorgia (Talk) 18:20, 26 October 2007 (UTC)
Does PR, as discussed above, need an "event" link? Perhaps it should cease being an event. Marskell 21:53, 26 October 2007 (UTC)
I think this is an interesting question. Part of the big picture of our discussion is that we believe PR should be separate from the article quality hierarchy: it should be something that any article can use at any stage to facilitate improvement, not necessarily to go for GAN or FAC or for any other reason. On the other hand, peer reviews can be really useful information for reviewers at these other processes. As originally constructed, ArticleHistory and PR were aimed at FA, but then again, GA was originally aimed at short articles. Times have changed. If we are reevaluating what PR, GA and FA are about, then maybe we need to reevaluate what AH is for. Geometry guy 22:33, 26 October 2007 (UTC)
I'm kicking myself for not asking the same question. I've frequently stated my view that PR is outside any hierarchical system, but I hadn't related that to the archiving issue until now. --Malleus Fatuarum 22:43, 26 October 2007 (UTC)
Good question: OK, what happens if PR isn't linked in ah? Either it gets lost in talk page archives (bad) or it generates another template on the talk page (bad). Do we want to lose PR info in articlehistory? Suppose an editor comes along two years after any other ah event has occurred, and wants to bring an article to FA status? Where would that editor begin? By looking through all old events that might provide useful tips for improving the article, and PR is one of those. SandyGeorgia (Talk) 22:49, 26 October 2007 (UTC)
In what way would a two year old review be likely to be helpful to an editor looking to improve an article to FA, or any other status? Just look at at a two-year old FA article. Many of them wouldn't even get through a current GA review. --Malleus Fatuarum 23:01, 26 October 2007 (UTC)
Completely varies by article; in one FAR, current editors were able to reconstruct the info they needed from info left years before by a long-gone editor. It's never a good thing to lose good info, and some PRs have good info. SandyGeorgia (Talk) 23:14, 26 October 2007 (UTC)
  • Maybe so. But as the lawyers say, bad cases make bad law. --Malleus Fatuarum 23:52, 26 October 2007 (UTC)

Don't want to interrupt the flow, but ...

I'm raising this issue in a separate section so that it doesn't get lost in the noise.

I would like to know why it's important to have an n-year old review available to an editor without that same editor also having available the n-year old criteria that article was assessed against. History always has to be viewed in context after all. --Malleus Fatuarum 02:10, 27 October 2007 (UTC)

I´m not entirely convinced that it is important. I don´t see much use to an archived peer review after two years. However, you can argue that it´s always better to be able to find something than not, so the separate page has an advantage. I haven´t thought that where it occurs is the critical issue. In terms of driving reviewers, having the list of people to contact on user talk has been the innovation being argued for. Marskell 14:03, 28 October 2007 (UTC)

On review locations, subpages, bots and categories

I don't really see how moving a peer review to some other page will improve participation. At least, it's not obvious that it will, and if the argument is that it's "worth a try", that's a mighty big change to the PR system just to try something marginally different.

All the featured processes use WP: subpages. The oldest were PR and FA, the rest (FL, FP, FT, FPO) probably copied them. I'm not really sure why they use the archive system for subsequent entries rather than the AfD numbering system. They could have just as well used permalink archiving and kept it all on one page. Guess that just indicates common ancestry.

The benefit of WP: subpages is that if an article is renamed, the associated pages in WP: space don't need to be moved. WP1.0 used a subpage of the article (at /Comments) for reviews. This is hardly ever used, but if it were, these pages would probably need to be moved with the article.

It doesn't really matter where something is as far as linking is concerned. A bot can deal with most anything. Let's not get bogged down in ArticleHistory - that template evolved in ways not originally intended.

If you want the PR page to "just be a list", then why not just use Category:Requests for peer review? That's a list of pages with requests for peer review, no bots involved. If you wanted to categorize them into 10 groups, you could have the template do that too. Gimmetrow 02:53, 27 October 2007 (UTC)

It's not "some other page", it's that article's talk page. Big difference. --Malleus Fatuarum 08:30, 27 October 2007 (UTC)

Checking on PR in article talk space

I would like to revisit an issue that we had consensus on. SandyGeorgia has commented that she opposes PR taking place in article talk space. Previously there was a definite consensus on this: see here. However, in the light of the above discussion, I'd like to get some input on the following options. Please note your comments under each option, as we have been doing.

PR takes place on article talk, not in a subpage. If ArticleHistory can refer to the PR, it probably requires a bot to copy it; Jayron's idea, above, is a variation on this approach.

  • It wouldn't be very much trouble to remove the completed peer review section from an article talk page and archive it to wherever it needs to go for a permanent link in the AH template. Reviewing on the article talk page seems to me to be a less formal, more open and more friendly approach. Along with others I also believe it might encourage more community contribution, simply because it's more accessible, which is my main reason for preferring this option. SandyGeorgia has raised some important considerations, but consensus seems to be that the current system does not work. Something obviously has to change - I do think this is worth at least a trial. EyeSereneTALK 14:17, 26 October 2007 (UTC)
  • Agreed. Rather straight-forwardly solved objections based on a perceived need to provide permanent links have not dissuaded from my earlier stated opinion that conducting the review on the article's talk page is likely to encourage a more collaborative review than at present. And encouraging collaboration is a far important issue, that what happens to the review after it's been completed. --Malleus Fatuarum 14:26, 26 October 2007 (UTC)

PR takes place on a separate, permanently linkable subpage of article talk


PR takes place on a separate, permanently linkable page elsewhere, using existing PR page naming conventions

  • PR stays where it is, named as it is. Why swim upstream? Moving peer review will require community consensus, and I don't think that will be easy. SandyGeorgia (Talk) 03:56, 26 October 2007 (UTC)
  • Support, this is the only way the semi-automated PR script will work as currently configured. Ruhrfisch ><>°° 04:04, 26 October 2007 (UTC)
  • As pointed out above, this page is not permanently linkable, but requires editor work (currently done by SandyGeorgia and Gimmitrow) when peer reviews are archived. Geometry guy 06:56, 26 October 2007 (UTC)
  • I'm not parsing, or maybe we're talking past each other Gguy; I'm not sure what your post is saying. The manual work is required in the rare instances that another editor goes out of their way to mess up an old peer review by completely overwriting it with a new peer review. An editor really has to go out of their way to misread instructions and standard procedures to do this. The links are permanent; perhaps we're using terminology differently? SandyGeorgia (Talk) 17:44, 26 October 2007 (UTC)
According to the instructions at WP:PR, when a new peer review request is made for an article that has already had a peer review, the old peer review is moved from Wikipedia:Peer review/ARTICLE NAME to Wikipedia:Peer review/ARTICLE NAME/archive1. However, the ArticleHistory link for the old peer review will be to Wikipedia:Peer review/ARTICLE NAME, and so, unless an editor or bot changes it, it will link to the new peer review, not the old one. For example the ArticleHistory at Asteroid belt links to the current review, not the previous one. Geometry guy 18:49, 26 October 2007 (UTC)
Right, we're on the same page so far. But whenever an editor moves a file, they're supposed to check and update links and redirects (I believe "Move" makes that clear). Articlehistory is supposed to be updated as part of that process. If the editor didn't do that, the advantage of standard naming conventions is that subsequent editors know exactly where to look for the old PR, it's easy to find, and they can easily update articlehistory if the original editor didn't do it. (Wish there was a way to get people to read instructions, but there's not, and that's an issue throughout Wiki.) The problem occurs when, instead of moving the old peer review, an editor overwrites it (who *does* that kind of thing anyway?); that is harder to correct, and since I'm not an admin, I have to correct them via cut and paste. Ok, so we're on the same page so far, and I'm still not seeing the problem ?? (BTW, I'll go check Asteroid belt and fix it, but that's something anyone can do, the editor who set up the neew peer review should have done, and regular editors should do. The same problem existed with the older templates.) SandyGeorgia (Talk) 19:40, 26 October 2007 (UTC)
Actually no: "move" tells editors to fix double redirects. Fixing redirects is way too much work for most page moves. And I found the example by searching alphabetically: it begins with an 'A'. I absolutely agree with you that not reading instructions is a problem throughout Wikipedia, and this is one reason I totally agree with you about instruction creep (WP:CREEP is quite explicit about this point!) Before finding asteroid, I also noticed that, contrary to instructions, some editors move the peer review to an archive at the end of the review: this is actually a better practise, because the archive is then a permanent link. I hope you get my terminology now: permanent means "does not need to be changed, ever!" :-) Geometry guy 20:33, 26 October 2007 (UTC)
That's one of the hidden beauties of GimmeBot; it pre-emptively moves the files to archive to leave the new file prepped for the next submission, so that subsequent editors theoretically have no chance to screw it up (I still check every FAC, and some still manage to find the strangest ways to mess up). This is in place for all FAs and FFAs, and he's chipping away at old PRs as they come up. Maintenance at FA, FAC, FAR and FFA gets easier by the day, as more and more are converted; there is little room for error now. OK, next question for clarification then: when you say "permanent" meaning never ever needs to be changed, are you saying that articles that have, say, five peer reviews would have them all on the same page instead of in archives? That would quickly become unwieldy. Maybe I'm still not understanding; each PR is a separate, archived event, and running them all together I don't think would work ... SandyGeorgia (Talk) 20:49, 26 October 2007 (UTC)
Yes, bots can sort out or avoid all sorts of problems, and GimmeBot is a particularly useful one. But it is better to design the process so that there are fewer errors to fix. Anyway, as for the clarification, the question of permanent links is completely independent from whether the reviews are on the same page or not. As long as each individual peer review never moves from one page to another, the link is permanent. I raised the idea of listing reviews together as a way to make it easier to browse through several reviews. Another option is to have reviews link to each other; this already happens informally, but it would save some work if it was automatic. Anyway this is all detail and I'm not wedded to any particular solution. Geometry guy 21:47, 26 October 2007 (UTC)
GimmeBot is moving all ah events that direction; that is, it moves the event to archive, and clears the redirect while leaving a link to the previous event, so that you can track back through events via the links left. The only events he can't do that with are GA events, since they don't have the separate archived pages :-) Another reason I'd like to see GA move the direction of all other events. SandyGeorgia (Talk) 22:46, 26 October 2007 (UTC)
That is certainly the right way to do it, but isn't the current process according to the instructions. The instructions need to be fixed anyway, because they don't explain what to do if there is already an archived peer review. Geometry guy 19:34, 31 October 2007 (UTC)
  • These proposals are confusing to me, but I think I am commenting in the right place. I believe I may have been the initiator of the controversy, as I may have been the first person to propose moving PR to the talk-page. My reasoning was 2-fold:
  • The problem I have with PR is that the page is SO HUGE that it is initimidating and hard to contribute. I enjoy reviewing articles a lot (as SandytGeorgia says, I find it "fun") but the sheer size of the PR page makes it hard for me to contribute in a useful way; I have a hard time finding articles I can contribute to constructively. I contribute often to GAN reviewes specifically because the way that GAN is orgainzed (article titles only, categorized by topic) makes it easy for me to find articles that match my interest and expertise. My thinking was that if PR only had article titles rather than full reviews, it would be easier to contribute and would solve the major problem of being overburdened.
  • Given the problem I saw above, I proposed moving the discussion from PR to the talk pages since a) that would unclutter the page and b) the article talk-page seemed a logical place to carry on discussions on improving the article. From my perspective, information about improving an article should be as close, in Wikispace, to the article itself in the interest of openness and collaboration. However, I can see that SandyGeorgia has a valid concern over the move to talk pages that I did not forsee.
Given ALL of that, would a reasonable compromise be to take the better part of the proposal (reducing the main PR page to an easier-to-navigate list of categorized links) and instead have a separate "Peer review page" for each article? I believe that is what this section is proposing, and even if it isn't, that is what I am proposing. SandyGeorgia, does that seem a reasonable thing to do, from your perspective? Anyone else? --Jayron32|talk|contribs 18:33, 26 October 2007 (UTC)
Clarifying still, each peer review does have a separate page now, so I may not be following still gosh, all this terminology is a killer. *If* what you're saying is to leave the actual pages alone (that is, each PR currently has a page with a standard naming convention) and then categorize the list so people could search in their area of interest (it's not the way I use the page, but I recognize it may be the way others use the page), I'm not crazy about the categories idea, but in the interest of compromise, I'd be in favor. The downside that I'm concerened about is that as soon as you start categorizing pages (as in the case of GA and GAC), you end up with top-heavy hard to negotiate hard to add to hard to edit pages that *decrease* participation, IMO. I am seriously concerned that some of these proposals are duplicating some of GA weaknesses (those convoluted pages which a gazillion categories that take forever to scroll through, so I just glaze over and click out). In the interest of compromise, I can be convinced, but the real question is what the broader community will accept, and I'm not sure the GA model enjoys broad support. Convince me; I like compromise :-) SandyGeorgia (Talk) 18:43, 26 October 2007 (UTC)
Re the categories, it is very important to emphasise that we are only talking about ten categories here, not the umpteen hundreds of categories that there are at GA! (I think SandyGeorgia may have misunderstood that above.) Geometry guy 18:54, 26 October 2007 (UTC)
That's why I said I'm open to compromise; on the other hand, experience shows that once you set up categories, everybody's got an opinion, and they morph, and morph, and grow, and grow ... there was an extended issue at the talk page of WP:FA, an agreement was hammered out with a lot of acrimony but that *all* parties agreed to although the editors are now in an ArbCom case, and someone <grin> changed that hard fought compromise just the other day :-) SandyGeorgia (Talk) 19:40, 26 October 2007 (UTC)
Really, there aren't full discussions listed on WP:PR? Because when I open WP:PR, I see the full text of every discussion, which is THE BIGGEST PROBLEM. I know that this is merely the text that is transcluded from the separate PR page, but that doesn't solve the main obstacle for contribution to PR, which is: Editors cannot find reviews they want to contribute to without reading through EVERY single review that is there. See, the problem is not that people don't want to contribute, it is that people can't find places to contribute because they get lost. Instead of transcluding the full discussion, why NOT simply list the link to the place where the discussion is happening? And I agree, GA handles categories WAY WRONG for our purposes here. However, to throw out the baby with the bathwater also seems a bad idea; what we need is properly managed categories, ten and ONLY TEN categories, with no subcategories, seems reasonable and easy to manage. The issue with GA (and to a certian extent FA) is that there are thousands of rated articles on those pages; the 10 top-level categories would have well over 100 articles each in them, which, well, makes categorizing pointless. However, at PR, there would be something on the order of 100 articles with actively open PR requests, which would result in an average of 10 articles per category (probably vary between 0-20 or so depending), which seems like an easily managable number. --Jayron32|talk|contribs 20:02, 26 October 2007 (UTC)
We may be using terminology differently, Jayron, but I think we mostly agree. I certainly don't read through every PR; I look at the Table of Contents for articles that interest me. In fact, come to think of it, often I specifically do not look at comments made by others (on PR or FAC/FAR), as I don't want to be influenced. I don't speak "transcluded pages" and all that, but I've not disagreed with the idea that we the main PR page could be only the list; I just don't know how that is accomplished on implementation. I don't think having the text there is a problem, but I'm not opposed to doing something to make it go away and convert it to a list (that's where we agree). I would also agree with 10 categories being manageable, but I highly doubt they'll stay that way over time; just look at how the GA page has gotten more and more convoluted over time. I suspect that the divisions on GA started out simply, as well. Anyway, we agree on 1) a list at PR and 2) no more than 10 categories. Where I disagree is moving peer reviews to talk pages or changing the long-standing naming conventions of peer reviews. SandyGeorgia (Talk) 20:12, 26 October 2007 (UTC)
Good, we are in full agreement then. I see your point on the talk page issue, and I am starting to see your point. It looks like we have solved that sticking point. --Jayron32|talk|contribs 20:32, 26 October 2007 (UTC)
Cool beans; do others like it? SandyGeorgia (Talk) 20:49, 26 October 2007 (UTC)

The beauty of the WP 1.0 hierarchy is that it is very difficult to change it, because it is used in multiple places, including release 0.5. This is completely different from the situation at WP:FA, where the structure of the page is determined by irrelevant and unedifying arguments about whether parapsychology is a science or not (and I will comment further there at some point!). There is simply no case for using any more than the top level of the WP 1.0 hierarchy at PR. I also believe that there is no case for using more than the top two levels at WP:GAN. Such a change would dramatically reduce the fragmentation of that page! As for the small areas of disagreement, I think there are multiple possible solutions, but that may be too fine detail for this ideas factory. Geometry guy 20:24, 26 October 2007 (UTC)

Agreed. As to whether parapsychology is a science or not, we can discuss that elsewhere, at some other time. --Malleus Fatuarum 23:10, 26 October 2007 (UTC)
I just looked at the WP 1.0 example per Parapsychology, and WP 1.0 presents numerous inconsistencies and ambiguities; not liking it at all. The compromise reached at WP:FA (based on the Dewey Decimal System) worked much better, IMO. Not saying I'm opposed to 1.0, but it's no panacea and doesn't solve the parapsychology dilemma adquately (discussing separately with Gguy at the talk page of WP:FA). SandyGeorgia (Talk) 23:16, 26 October 2007 (UTC)
Err, I didn't see any "dilemma", and I don't see that a discussion that confuses science with the scientific method has any bearing on this present discussion. --Malleus Fatuarum 23:40, 26 October 2007 (UTC)
This discussion only concerns the top level in the WP 1.0 hierarchy, which is pretty stable and widely used. Geometry guy 23:45, 26 October 2007 (UTC)

PR takes place on a separate, permanently linkable page, and the location doesn't matter

Template:ContentReviewMedal

People were talking about the need for an award, so I made Template:ContentReviewMedal. It's not only for PR; can be used for FA GA PR etc. The image probably needs tweaking (soften the edges of the silver wiki...?]. The template probably needs tweaking too. Tweak away; I shouldn't be doing this anyhow... should be working on dissertation...  ;-) Note to self: back to work! Mush! Yah mule! --Ling.Nut 07:54, 28 October 2007 (UTC)

Hey, well that´s one thing done. The magnifying glass seems rather oddly tacked on, but otherwise I think it fine. Marskell 13:59, 28 October 2007 (UTC)
Yeah, nice one - I'll take carrots over sticks any day. cheers, Casliber (talk · contribs) 14:10, 28 October 2007 (UTC)
Magnifying glass: I had a pretty hard time thinking of anything to symbolize "reviewing". It wouldn't fit nicely on the silver wiki medal (looked too cluttered/busy), so I had to stick it on the ribbon ;-) Thanks for the kind words --Ling.Nut 14:48, 28 October 2007 (UTC)
Very nice idea; I often have to choose the Working Man or Diligence barnstars to award folk who have done a lot for FAC or FAR and don't feel those awards sum up the effort; this would be much better. Now, BACK to work on the dissertation, Ling.nut!! SandyGeorgia (Talk) 15:32, 28 October 2007 (UTC)
Hey Ling nut, coulda done a nice image of a masseur - the masseur medal for prose massage, or a wikimicroscope for minute copyediting...cheers, Casliber (talk · contribs) 20:26, 28 October 2007 (UTC)
Although...as mentioned, we already have a Reviewer's Award. I notice that it's only ever been given to five people. One was me, and two of the others were people I gave it to... So the problem perhaps is not the lack of the award but the lack of propagation. Marskell 16:14, 29 October 2007 (UTC)

Proposal: Ban automated peer review bots

For a first proposal to help improve the peer review system initially, I would suggest banning the use of all of the automated peer review bots. I don't think anything truly useful really comes out of them, as they mostly suggest manual of style and really basic grammatical changes, many of which don't even apply to the article in question. They're also very useful. When people come to peer review, they want an analysis of the article's content, and any information that is potentially missing in order for the article to be improved, as well as suggestions how to actually improve the article's prose. None of the bots really go into these details all that much, and mainly just concentrate on the mundane grammatical and wikifying issues -- in which case, a better solution for the users of the bots is to take the information that is fed out from the bot and just make these minor updates themselves.

I also think that the high number of automated reviews at WP:PR can also be discouraging human reviewers. Many reviewers might see the automated bot review, and say to themselves, "well, a bot has already checked this out, so there's no point in me reviewing it, since I'll probably just be rehashing stuff that the bot has already said." Which is really a bogus excuse to not review an article, but an excuse nonetheless. The human reviews are the most useful information here. The only real use of a bot might be to reconfigure it to strictly check for manual of style issues, and to run that at the conclusion of a major review for something like WP:FAC, just to make one final check that the article is in good shape. Dr. Cash 01:38, 12 October 2007 (UTC)

I have no problem with stopping automated peer review bots, but I disagree with most of the rest of your argument. Peer review involves looking at an article's neutrality, comprehensiveness, etc., but it also includes all the little manual of style things that are easy to overlook. I see no problem with individual editors running the peer review script on articles and then posting the results. Often, the bots find lots of errors, and it's unfair to expect an uninvolved editor to make the changes themself. The time I have to devote to Wikipedia is in short supply, and it's frankly better spent elsewhere. — Brian (talk) 01:46, 12 October 2007 (UTC)
I'd pretty much go along with Dr. Cash here: any quality review process needs itself to be producing reviews of a decent quality - for reasons of credibilty if nothing else. Bot reviews all generally seem to say the same things regardless of the article under review, and come over as impersonal, unhelpful and give the impression that the article hasn't actually been looked at by an interested reviewer. Seeing these banned across the project would be only be a 'good thing' IMO (unless perhaps retained for use by editors themselves to check articles before submission). EyeSereneTALK 10:28, 12 October 2007 (UTC)
I think the automated reviews can be very useful, if used sensibly and in the knowledge that it's not infallible. That is only the case for a mind that is genuinely looking for ways to improve an article, though. Given the amount of advice given at peer review that is ignored by those who requested it, I suspect that not all editors are looking for opportunities to do more work! That being the case, we might as well just have a note at the top of the peer review and/or MoS pages noting that there is a tool that can automatically review your article. I see it as along the lines of a spell or grammar checker. (Hey - maybe we should require editors to run their articles through a combined spell, grammar and MoS checker before they submit to FAC!) 4u1e 16:20, 12 October 2007 (UTC)
I've never really found the bots that useful. I can always see why they have made the mistakes they have, but that doesn't help me improve the article, and I am dedicated to improving articles. I also find the impersonal nature of the bot off-putting. Editors who put hours into articles are, generally, I find, not happy to find a bot telling them not to use "weasel words" or whatever. Those decisions are context-based and our bots have not reached that level of sophistication. I am in favor of banning the bots. Awadewit | talk 02:27, 13 October 2007 (UTC)
The benefit I do get from them is as a reminder of various MoS points I may have forgotten about. As I say, along the same lines as spelling/grammar - it's not that I karnt spel, but a tool to catch my goofs is useful. I agree that it's probably better not to use the bot on the Peer Review page. 4u1e 15:48, 13 October 2007 (UTC)
Eye wood bee won off those inn favour if banning bots. --Malleus Fatuarum 16:11, 13 October 2007 (UTC)

One possible idea regarding bots. Maybe the best thing would be if a script of some sort was created and available that would check the wiki-code of a page for basic grammar and spelling, as well as for major issues with the manual of style. But instead of putting some generic suggestions in some type of archive of the review page, the script would highlight all of the pertinent issues and display that to the user directly, allowing the user to make the necessary changes themselves. I would think such a script could be coded in PHP, making it easy for any editor to access it via a web page to check their own article for these basic issues, and change it themselves based on what the bot suggests. We probably don't want to make the script automatically update the page with all of its suggestions, because many "issues" that it finds might actually be cases where the manual of style might need to be ignored for one reason or another. But I think it would be far more helpful if users ran the script themselves and saw all of the specific issues right there, rather than having to read through the article based on a very non-specific description of all of the "issues" in the article. And of course, we'd still eliminate the automated reviews from GA, PR, & FA. Dr. Cash 19:11, 13 October 2007 (UTC)

That's a great idea, I think. It would allow a lot of those minor issues to be corrected prior to nomination. The better the articles are when they come to us, then faster the review process can go. Lara❤Love 14:04, 15 October 2007 (UTC)

A dissenting view

I currently run the User:AZPR account (with User:AndyZ's blessing and permission). It is not a bot, it is a semi-automated tool that still needs a human user. I go in and open each file and cut and paste the suggestions into a file, which I then paste into the current WP:PR/A page. I then go in and add the automated peer review notice to each article on the WP:PR page, which is one line if done this way ("Please see automated peer review suggestions here. Thanks, APR t 02:29, 23 October 2007 (UTC)"). The script does add this notice to all files at one go, which saves some time.

If your goal is to get more participation in PR, then I fail to see how eliminating this helps. If done in this way, it adds one line to the peer review, so I doubt anyone thinks "No need to review this one". I also find it very helpful for catching lots of nuts and bolts issues (MOS especially). Is it perfect? No. Does it help? I think so. Are you free to ignore it? Of course. I have occasionally seen a request not to use it and I honor those (less work for me).

There are a number of other editors who also run the script and paste the file directly into the Peer Review. I find this less helpful, because it makes it look like the article has recieved a substantial review (as the automated peer review can be a large number of lines of text). I have some other ideas for improving PR, GAC, and FAC, but will add those below. My final comment is that while it is great to suggest another bot, who will write it? Ruhrfisch ><>°° 13:16, 23 October 2007 (UTC)

PS I have never seen the automated peer review tool used for GA or FA nominations, nor do I ever run it there (unless asked, when I pate the review into the article talk page). I agree it has no role there, but if a GA or FA nominee has real issues which the script catches (MOS usually) then it is not ready for FA or GA status. Ruhrfisch ><>°° 13:20, 23 October 2007 (UTC)

In support of the above, but also kind of in support of the previous views: I use the semi-auto reviewer personally, on articles I've written, to check various MoS issues. Generally, I've found it helpful for spotting things I've missed, or wasn't aware of. I've also used it to check articles I've GA reviewed, but I've never just posted its findings; in all cases, I've checked what it came up with and either ignored its findings, or found the specific problems and either fixed or mentioned them in the review section on talk page. Of course it makes mistakes, and I wouldn't ever dream of posting its results as a GA/FA or any other kind of review, and so I fully endorse the banning of it in formal peer reviews, but I'd hate to see it denigrated to the point it's no longer used at all. Carre 17:49, 23 October 2007 (UTC)
What do you mean by "banning of it in formal peer reviews"? Just in FAC and GAC, or in PR too? While anyone can add the script to their monobook, a lot of users don't know how to do that or don't want to, and I get some requests to run the script from people who know I have it installed. Ruhrfisch ><>°° 15:08, 24 October 2007 (UTC)
I agree with you: I think semi-automatic reviews are fine as long as they are stored on a separate page. Concerning the final question, the idea is to keep the bot pretty simple, so that a bot request will be quickly answered, although I can also think of a couple of editors whom I could ask. Geometry guy 16:42, 24 October 2007 (UTC)

I think it's great if people want to use this semi-automated program to find a lot of these MOS and grammatical issues and fix them personally. If more people did this, then the quality of articles being submitted to review would be much greater. But I find that simply posting something like, "Please see automated peer review suggestions here." as the only text of your review to be very impersonal. It sort of says, "I didn't have time to actually read your article, but I ran this automated script, so here's the results." That's not exactly a good way of doing a review. Dr. Cash 18:33, 24 October 2007 (UTC)

So if I understand your point of view, you think the semi-automated suggestions are useful if run by the editor themself, but not if run by me or another editor? With all due respect, I do not think the semi-automated peer reviews are to be confused with regular peer reviews, which is why they are just one line linked to the actual suggestions on a separate page (WP:PR/A). I would love to read all the articles and make comments on them, but I do not have time for that. There were 109 semi-automated reviews in September (on WP:PR/A, plus others that were left in the review itself by different editors). In October so far (one week to go in the month) it is already up to 112. Ruhrfisch ><>°° 19:03, 24 October 2007 (UTC)
I suspect this can be resolved by a simple rewording, such as "A script has been used to generate a semi-automated review of the article for minor issues of grammar and house style. If you would find such a review helpful, please click here." Geometry guy 18:57, 24 October 2007 (UTC)
Was taking a day off WP yesterday, so didn't spot this lot. I think the above bits have pretty much explained my opinion. The semi-auto thing is useful, but it does make mistakes. Simply taking the time to check its results is much more personal and helpful than just pointing an editor at its output, especially as it does have known issues. So, it's useful whoever uses it, but taking the extra step of checking it and humanising is so much better. I know that if I submitted an article for PR and only got the bot's output, I'd feel pretty disheartened by the whole process. Carre 11:19, 25 October 2007 (UTC)

Although I know a couple of editors have suggested that use of the semi-automated review be reduced or changed to be request-only, I don't think there's a consensus in this workshop for those changes at the moment, so I don't think it's likely that we will propose anything along those lines. That's not to say that conversations like this are not useful, just that I don't think this is something that the workshop is likely to propose, so we don't have to settle these questions now. Mike Christie (talk) 20:01, 24 October 2007 (UTC)

I support and approve the automated peer review bots; when I was a Wiki newbie, the info it provided was a helpful starting place. Many newcomers first encounter with any sort of outside input may be peer review, and the automated information may be helpful to them. SandyGeorgia (Talk) 01:11, 25 October 2007 (UTC)
I just run the script (and do not have access to it, or the expertise to modify it), but I will see if AndyZ can modify the text left on the PR page to be more user friendly as suggested above. Unfortunately there are some review requests that only get the semi-automated PR comments. If the script were not run, then they would presumably get nothing, which would be even worse. I do try to look at the oldest PR requests and make comments on the ones that received nothing but the scripted note. As for hand checking the scripted replies, the number done (well over 100 per month) and the nature of the script (when I run it as AZPR) both make that difficult to do. I would be fine with putting a note somewhere in the instructions that if a semi-automated peer review is not wanted, just say so. I do read the requests and skip those that ask to be left out. Ruhrfisch ><>°° 15:20, 25 October 2007 (UTC)
AndyZ has changed the wording of the message left in peer review to " "A script has been used to generate a semi-automated review of the article for minor issues of grammar and house style. If you would find such a review helpful, please click here." as suggested by Geometry Guy above. Thanks, Ruhrfisch ><>°° 13:49, 1 November 2007 (UTC)
I'd like to see "for minor issues of grammar and house style" either removed or changed. It doesn't just pick up minor issues, it also picks up very major ones such as lack of inline citations or references, and unsupported claims like "Some people say..." and "It has been suggested that..." DrKiernan 14:07, 1 November 2007 (UTC)
OK, how about just taking out "minor" and linking "house style" to WP:MOS? Ruhrfisch ><>°° 14:48, 1 November 2007 (UTC)
OK. Thanks, DrKiernan 14:52, 1 November 2007 (UTC)

Reinventing the wheel

After the glut of talk on the 25th and 26th, people who have been here for three weeks may suddenly have become exhausted. We really should have considered AH earlier, although I don't see it as a deal-breaker. Another thing I don't see is a real "PR community" that is going to be resistant to changes. On that note, can we move forward and suggest on WT:PR that the categories be placed on the page at least? As noted, I'm not perfectly happy with the ten but they are certainly better than an undifferentiated list of 200 reviews; we now have Outriggr's mock-up above. Both the talk page/separate page and reviewer list issues can be subsequently revisited. Where the review takes place has never, incidentally, been my main concern. A list of people to contact on user talk is the only innovation that I think will directly increase reviewing. (Another thought I had in looking at PR today, is purging those sub-PRs from the page that are clearly not working; this will raise the profile and value of those that remain, such as Science and MiltHist.)

On the subject of people dropping by out of nowhere, a question: does the (not perfectly defined) "Raul veto" extend to PR? Hmm. Marskell 16:13, 29 October 2007 (UTC)

I'd support Outriggr's markup being proposed and hopefully implemented at PR, but I'm not following the question on Raul's veto. Are you asking if his FA directorship extends to PR, so that we should bring him in? Also, there are some very regular PR reviewers (I can't remember, is Dr KiernanI am observing! DrKiernan 19:00, 29 October 2007 (UTC) participating in this discussion, for example?) SandyGeorgia (Talk) 18:00, 29 October 2007 (UTC)
I would also ask Allen3 who singlehandedly does almost all the archiving of reviews that are over one month old or have received no new responses in two weeks. I still think it would be useful to ask requesters to specify what they plan to do after PR (choices are FAC, GAC and general article improvement). Even if it is not required, it might also be good to ask all requesters to also review two other articles in the PR queue. I also like the list of oldest articles without a response section of GAC. Ruhrfisch ><>°° 18:21, 29 October 2007 (UTC)
Yes, that's what I meant regarding Raul, though I don't think he takes much direct interest in PR beyond updating FAS. Dunno. And duly noted about Allen3. It occurs to me that the mistake we made here was not broadening things after gaining consensus on Awadewit's initial list of five. We then travelled too far down the details path without considering vested users who might have opposed viewpoints. Marskell 18:30, 29 October 2007 (UTC)
On the Raul question, my guess would be that he's tied up enough in the FA process that he might not have an interest in the nitty gritty of PR. As to asking nominators what they plan to do after PR, I can see that it would be helpful, but every article should really be aiming to meet FA standards anyway, so I review them all according to same. I make no distinction in my review for what stage of the process the article is at. I'm always wary of requiring users to do anything because 1) it rarely works (note pet peeve about asking nominators to do notifications at FAR <sigh>) and 2) a bad review (that is, someone who really doesn't understand the criterion) can be worse than no review. I've seen too many deficiencies come through FAC that are there because "another reviewer told me to do that". We want knowledgeable reviewers. On the other hand, this isn't a sticking point for me and I wouldn't oppose it if it's worded rather weakly. SandyGeorgia (Talk) 18:54, 29 October 2007 (UTC)
There has been a lot of agreement here that PR should be a general purpose process, and an emphasis on prepating articles for FAC is not helpful (although that is one very good use for PR, and, in a sense, all improvements do move the article towards FA quality). I think that answers the "Raul veto" question (there isn't one). I agree it is useful if PR nominators indicate their plans: this is something that could be suggested rather than required. I also agree with Ruhrfisch that there is no harm in asking nominators to comment on other articles, but again, I would not want this to be a requirement. Geometry guy 22:56, 29 October 2007 (UTC)
If there is agreement to present a mock-up, I hope we can do a better job: present a template that produces something like the line currently shown under Arts, with the modified instructions necessary for users to achieve a listing. (I can envision more informative visual sugar on this main page, like offering the requestor's intro text in a "hidden" state, a bot that updates the PR listing with how many users have responded in each PR, etc.) I'd volunteer to fill out the mock-up to be more appealing and realistic, but it would probably be painstaking for me. Any page designers in the crowd? I'm also not clear if the workshop community is ready for such pragmatics yet. –Outriggr § 23:07, 29 October 2007 (UTC)
The point of the Raul veto question was, in part, to point out that we need to be thinking about users who have given time to the processes and who might appear and trip this up. For instance, we might ask "why did Sandy crash the party a few days ago?"; conversely, we might observe that Sandy and Gimme (through the bot edits or his programming of it) have almost certainly spent more time on process maintenance than anyone else commenting. So again, I think we made a mistake keeping the group closed.
I agree that, theoretically, all articles should be considered with FA as the final goal (that GA handles articles FA cannot remains undemonstrated). Very clearly, people do often initiate PRs with FAC as the next step in mind. But I also agree that PR is simultaneously a general purpose process. That is, FAC is an "invisible goalpost" for everything, but in the midst of a review its criteria need not be mentioned. Thus I would oppose "list your next step" as a requirement.
Anyhow, I am going to start a PR thread later today unless some clear and present danger in doing so is presented. The usefulness of remaining only on this talk has been exhausted. Marskell 09:38, 30 October 2007 (UTC)
As one of the involved parties, I can assure you that I will not attempt to veto the changes proposed here. I will also not serve as an enabling force for the effort. My previous comments on the issues raised on this page are available at Wikipedia talk:Peer review/Archive 2#Subpages, Wikipedia talk:Peer review/Archive 2#Redesign needs to consider complete process, Wikipedia talk:Peer review/Archive 2#Has the division of the page had any positive effects?, and Wikipedia talk:Peer review#Dead?. --Allen3 talk 15:04, 31 October 2007 (UTC)
These links to previous discussions and your comments are very helpful. I especially like "Redesign needs to consider complete process". Indeed! While I sympathise with those here who want to go out to the content review processes and implement our wonderful ideas ;) I have repeatedly argued against doing this. As Allen3 rightly points out, categorizing PRs without making any other change will add to the administrative burden of maintaining the PR page. For the idea to work, it needs to be coupled to other ideas which reduce that burden, such as automating the page entirely. I guess the main maintenance job is (quoting Allen3's edit summary) the "Archive of peer review requests that have received no new responses in last two weeks" (and replacement of {{peerreview}} by {{oldpeerreview}} on the corresponding talk pages). This is clearly a job that could (and I would argue, should) be done automatically. Far too much routine maintenance such as this is done by dedicated editors such as Allen3.
But returning to the big picture, all of our ideas have impacts, and ideas which address one of the problems we have highlighted will have knock on effects elsewhere. This workshop has so far been successful at getting people involved in different aspects of content review to work constructively together. For it to achieve any more than this, we really must resist the urge to implement, and instead view this workshop as an ideas factory, not a "provide solution x to problem y" factory. Geometry guy 19:11, 31 October 2007 (UTC)
But that runs into the contrary problem of attempting to construct an entire house of cards prior to any real achievement—we invested three weeks and it toppled over. We do that again and this workshop is dead. I do not see anything insurmountable presented in the links above (though I appreciate Allen has spent much time with maintenance there). In fact, I see a lot of good editors who broadly favour the idea of categories. (Sub-pages no—that really would be too much maintenance.) Marskell 19:28, 31 October 2007 (UTC)
I had thought of moving to the volunteers list first, incidentally—but if we want people to detail their interests based on categories, we'd have a cart and horse problem. Marskell 19:35, 31 October 2007 (UTC)
... and changes can often have unintended consequences. Which is one reason why I have never been happy with the effort to sort everything out rather than focusing on one issue at a time. Iterative and incremental always beats waterfall IMO. --Malleus Fatuarum 19:37, 31 October 2007 (UTC)
No it didn't topple over. There was useful new input, and the issues with article history admit multiple solutions. And no, we are not constructing a house of cards if we generate ideas rather than solutions. And no, we don't sort things out, we generate ideas. This workshop cannot sort things out, because it is just a handful of editors, not the community. Bringing in the community is not the solution to that; recognising the limitations of our small scale is. Geometry guy 20:09, 31 October 2007 (UTC)
G'Guy, if you mean to suggest that this workshop will only go over ideas and never pause to try to implement any, then we have very different priorities here. You're asking for a totally blind investment of people's time: let's all talk but there's no assurance anything is going to actually happen. That's not on. Marskell 09:16, 1 November 2007 (UTC)
I think we have the same priority: improving the content review process. Of course that means communicating our ideas as well as having them: if they are good, they will be implemented, but this workshop can't actually implement ideas, and it is no use pretending we can. What assurance do you want? "The Facillitator assures participants that something will happen"? If this is a blind investment, then so is Wikipedia: after all, any contribution can be deleted. We proceed on the assumption that good contributions will survive. So, no, this workshop should not pause, it should move on. Individuals within this workshop are welcome to disseminate any of the ideas generated here and see if they can be implemented, but I urge caution against doing so hastily. Some ideas lend themselves to an early implementation, some need more time to see their ramifications. It is not always obvious which is which. Geometry guy 11:16, 1 November 2007 (UTC)
What assurance do I want? I want at least some of the reforms implemented to assure me that I'm not wasting time. We're talking past each other slightly. Of course the fora that we've created here cannot literally institute changes—but I see no convincing reason why individuals cannot move, once ideas have a reached a certain degree of specificity, to the talk pages of the processes themselves. We don't need the entire community to add categories to PR. We need the people who help maintain the page and some editors experienced in content review generally (as most of us here are). Here's a quote from a participant:
"Archiving inactive discussions would certainly be easier to do manually with discussions all on one page, but it occurs to me it would be pretty easy to write a bot to automatically archive discussions that haven't had any edits for a week, or two weeks. As far as what the design should fundamentally do, I would like to see a design that simply encourages more reviewing. The fundamental problem with PR is that it's often the case that a request for review will get little or no response. I think that at least part of the problem is that for a reviewer, it's difficult to find articles you're interested in reviewing."
That was (the sorely missed) Worldtraveller, eighteen months ago. I fundamentally disagree with Sandy's "it ain't broke" theory regarding PR—all of the issues we're talking about now were clear in at least '05. I don't just want to jaw jaw. Thus, rather than "We cannot add categories because of extra archive work", let's say "how can we overcome the extra archive work problem?". Answer that, and then let's do it. If you don't want to agree with User:Marskell, at least consider Dr. Johnson below. Marskell 13:45, 1 November 2007 (UTC)

← I agree with most of what you say, especially the "rather than... let's say...". I also share your disagreement with Sandy's "it ain't broke". Some GA people say the same thing about GAN, despite the backlog of 200, and defend the instruction creep as being necessary to make the process work. I also agree that individuals should explore shipping out ideas, and that most ideas don't need wide input. I'm particularly keen on the automation ideas we have discussed, as I think they apply to GA as well as PR. As you say, this goes back a long way, and it is surprising that we still have routine tasks being performed by hand on a daily basis. I suppose I am only saying two things. (1) This workshop does not have authority, so moving ideas from here to other talk pages under the banner "the content review workshop thinks you should make this change to your process" has a high probability of back-firing. This workshop will only gain authority if the ideas it generates are good. So far impressions of this workshop have not been entirely favourable (Sandy's initial impression can be found on Mike's talk page, and it doesn't take much reading between the lines to get a feel for Allen3's view above). (2) Let's not rush this: our ideas may have flaws, and valid objections do need to be considered. I guess I actually agree more with Marskell than with Dr. Johnson. Geometry guy 20:21, 1 November 2007 (UTC)

That simply reinforces how right Dr Johnson was in what he said. The proposal has stalled because of a minor concern over article history, when the really important point was to encourage reviewer participation. Talk about the tail wagging the dog! --Malleus Fatuarum 21:04, 1 November 2007 (UTC)
Well, yea. Let's not say my entire impression of this page/workshop is negative because I came late to the process and found a couple of things that had been overlooked (possibly because of a small bit of skew in the representation of all processes). Also, I think peer review works when it works; sure, we need an automated process for archiving. My biggest beef was only that moving PR to talk pages would not be a good thing. Of course, I could never disagree with Samuel Johnson. Is the process stalled? Does anyone disagree with moving forward on suggesting changes at PR? SandyGeorgia (Talk) 21:17, 1 November 2007 (UTC)

An apposite quotation, to refocus minds

Nothing will ever be attempted, if all possible objections must be first overcome.

— Dr. Johnson
Touche. Marskell 09:16, 1 November 2007 (UTC)

mock-up

Sometimes it helps to visualize. I created Wikipedia talk:Content review/workshop/Peer Review mockup to visualize a main PR page that lists only the basic details of the PR request, and uses the "1.0" categories (the greatest consensus here seems to be that categorization is desirable). I'm no wikicode fanatic so it's pretty basic (and I just about gave up due to its likely triviality), but play around with it if you like. Of course, one entry on this page would ideally be created in a simple manner through the use of one (hypothetical) template. Since the template doesn't exist yet, it's harder to demonstrate anything...

Some other comments while I'm here. At first I liked the idea of the peer review happening on the article talk page, but increasingly I believe that there is value in the review happening in a "neutral place", as Sandy suggests. In general, when a process exists on wikipedia, it's nice to give it its own home and a sense of "place". (The peer review could even be transcluded onto the talk page while it was active as a further "FYI" for editors—beginning and ending a PR request currently involves editing the talk page anyway.) Originally I thought it obvious that the talk-page peer review activity would naturally draw in more commentary from editors, but based more on experience than idealism, I'm not sure that the talk page is really going to save PR (increase activity, I mean). We have 2 million talk pages now, and your experience is probably all you need to realize that most talk pages just aren't utilized much in discussions about content improvement, unless the article is popular, in which case it probably doesn't matter where the peer review is filed.

I am a big fan of categorizing peer review requests, but the difficulty of all article categorization on wikipedia (by subject or quality) is that the number of members of each category is generally horribly unequal—we're talking order of magnitudes quite often. So "Math" is one of the ten "1.0" categories? Great, but it will be an empty set (hehe) on the peer review page the vast majority of the time. On the other hand, "Arts" will pick up most popular-culture stuff and fail to adequately separate "fine arts" from the vast number of requests related to TV, bands, etc. None of this is a showstopper, but if we're going to implement categories, it would be really nice if the system were designed for once such that categories are somewhat similarly populated. (What proportion of current PR requests would fit into the 1.0 categories of philosophy/religion, social sciences, natural sciences, language and literature, and math, COMBINED? Scary.) –Outriggr § 06:24, 28 October 2007 (UTC)

"...it would be really nice if the system were designed for once such that categories are somewhat similarly populated." Yes. I really don´t much like these eleven for that reason (particularly Arts and Natural sciences alone) but appear to be in the minority. Thanks for the mock-up.
(Is it ´"order of magnitudes" or "orders of magnitude? :) Marskell 14:07, 28 October 2007 (UTC)
Oops... :) –Outriggr § 05:04, 29 October 2007 (UTC)
Nice work, Outriggr! I like it, a few notes:
  • I see you put "The oldest requests are listed first within each category." That's an interesting deviation from most other pages and could help solve the backlog at PR. (By the way, anyone notice that this famous backlog has dropped by about 50% over the last few months? I wonder if it was a summer in the northern hemisphere thing?)
  • Noted the aforementioned problems with 1.0 categories, but can't see a better way to address that (why *does* Math exist as a separate category in 1.0 anyway?) Again, I would scream if the 10 categories became a gazillion as they have on the GA page.
  • On the other discussions about how to help PR, then could we add to the PR page a transcluded signup list of areas in which editors are willing to help? I think it important to have a list of editors willing to do ref cleanup, MOS cleanup, copyediting, content review by area, etc. I envision a transcluded signup list page that would include the following groups:
  • General copyedit
  • MOS cleanup
  • Ref cleanup
  • Image review
  • Content review
    • Add each of the ten categories ...
  • and so on ... that's all I can think of for now ... I enjoy hitting random articles with sample edits of MOS and ref cleanup, because other editors can learn from the sample edits. SandyGeorgia (Talk) 15:28, 28 October 2007 (UTC)
Thanks Sandy. I haven't noticed the backlog reduction, but haven't been looking. Ideally, the changes proposed in this workshop will strengthen peer review regardless. I'd propose the new peer review page layout have an attractive, columnar format (transcluded by section with edit links), in which the peer review resources you mention could perhaps run along the right side, with the requests running along the left. Somebody who already knows wikicode well should mock that up! I agree about categories (I have absolutely no idea how the GAN category list has maintained "consensus" in its current, overzealous format). –Outriggr § 05:04, 29 October 2007 (UTC)
Dig it! This is what I had envisioned PR becoming. FWIW, I like the template under the arts section, which looks like a varient of the "la" template, with the review bolded. For me, that's the easiest to follow. I am not sure if we should include the list of reviewers on THIS page or make a prominent link to a separate page. I only raise a concern about it as this page is already getting busy enough, and we are approaching information overload on it. Maybe two pages (one for articles and one for reviewers) may be more appropriate. --Jayron32|talk|contribs 06:44, 2 November 2007 (UTC)

Looking at it from the other direction

This project has spent some time considering ways to increase participation in the various review processes. What about ways to avoid decreasing participation?

Having now received my second extensively abusive email from the editor who GA nominated the Albigensian Crusade, which I failed after a hold period of one week, I am beginning to feel that reviewers ought to be offered the same kind of protection that rugby or football referees are given. In any event, the experience has certainly made me think twice about continuing my involvement with the GA review process. --Malleus Fatuarum 02:04, 2 November 2007 (UTC)

  • I've said it before, I'll say it again: Failing GA is a nontrivial blow to the pride of editors, since GA really just means something vaguely along the lines of "adequate" or "sufficient" or whatever.
  • I myself had to quit GAR for a long while 'cause I took it very very personally when people criticized the GA process (in my own defense, the criticisms were not always collegial; sometimes far from it). people get a huge, killer, monstrous case of the Sour Grapes that Ate Manhattan... It makes reviewers drop out, and those that remain become very defensive
  • I think part of the training process (ahem) should be a heads-up that prepares newbies for this fact. Beyond that, mutual commiseration and the occasional barnstar are all i see necessary. --Ling.Nut 02:25, 2 November 2007 (UTC)
  • A problem with a single-reviewer system is that one person has to take all the heat. SandyGeorgia (Talk) 02:37, 2 November 2007 (UTC)
  • I used to do more peer reviews but frequently got no thanks and worse: no edits / changes to the article based on what I thought were valid suggestions. Not as bad as abuse, but neglect also does little to encourage reviewers. I am fine with moving on to a larger community on this, thanks for everyone's work. Ruhrfisch ><>°° 02:55, 2 November 2007 (UTC)
(edit conflict)I will admit that the experience did make me wonder about the wisdom of a single-reviewer system. But similar things happen with the collective FA review system as well. As we've probably all seen recently. --Malleus Fatuarum 02:58, 2 November 2007 (UTC)
<ahem> <smile> SandyGeorgia (Talk) 03:40, 2 November 2007 (UTC)
That's also a very good point. It's very discouraging to spend valuable time doing a good review when nobody else seems to care about the article. I've tried doing an initial mini review, just to see if anyone's actually watching. --Malleus Fatuarum 03:08, 2 November 2007 (UTC)
I used to try and do more reviews of the oldest articles that had no reviews, so maybe the requesters had given up. Still it was discouraging, Ruhrfisch ><>°° 03:17, 2 November 2007 (UTC)
One of the reasons PR is fun for me is that I can respond according to the nominators' interest. If I don't know the editors, I dip in tentatively with only some MOS commentary, and if they respond well, then I engage more with them and the article. I encountered one nasty set of nominators at PR once; they debated every single tip I offered them. So, I bowed out. No problem: we're not obligated at PR to strike/update as we are at FAC/FAR or GAN because we're not judging, only suggesting. When they came to FAC a month later, they got the same objections I tried to tell them, and they had some long rough going to get through FAC. On the other hand, I've met some very nice editors via PR, and helped shephard them through GA, FAC, and now even FAR. I've met more nice nominators than nasty, but maybe because I have selective memory on the nasty ones :-) One way to avoid burnout is to engage tentatively so you can gauge reaction. By the way, this is another reason I support having PR in a file in articlehistory. How discouraging to type up a lot of helpful commentary and not have that info available to future editors if the current editors don't finish the suggestions. It must be lonely to be a solo GA reviewer and take all the heat. Once, someone nominated three articles I had entirely written to GA without informing me, they weren't ready, and I gave those poor reviewers so much grief. Today they're Wiki friends. I just got a nice commentary on the Youngstown, Ohio FAC; the editors said they wish they had gotten this kind of help during their two peer reviews. When PR works, it can really make FAC or GAN smoother sailing. HTH, SandyGeorgia (Talk) 03:40, 2 November 2007 (UTC)

Going along with ways to avoid decreasing participation, WikiProject Good Articles just rolled out its first issue of a newsletter. It's being distributed via user talk pages to all participants of the project. Hopefully this will keep people in the loop and keep everyone updated on the status of backlogs and other such things. Dr. Cash 06:25, 2 November 2007 (UTC)

I hope someone is going to write an entry about how to add the GA templates to talk pages, and how to maintain {{articlehistory}} :-)) SandyGeorgia (Talk) 06:28, 2 November 2007 (UTC)
This was actually brought up on the talk page of the newsletter, and I agree, it is an important topic that I think I will try and get into the December issue. Thanks for pointing it out. Dr. Cash 06:53, 2 November 2007 (UTC)
This week, I corrected several GA templates, so when you get to it, it's not only articlehistory. SandyGeorgia (Talk) 06:59, 2 November 2007 (UTC)

(undent)<start broken record>.. and if your article about {{articlehistory}} is a sweet one, all or most of it could be added to the GA training program </end record> --Ling.Nut 11:25, 2 November 2007 (UTC)

Recap

Sorry I've been away from the page; not a good start for someone who is supposed to be the facilitator. I've been very busy off-wiki and I felt I had to devote the limited time I had to a FAC I had going.

I've tried to read through and catch up, and I've looked at the PR thread and Allen3's comments. Is this an accurate summary of where we are?

  • There's agreement that categories are worth trying, but Allen3's comments indicate it's pointless unless a bot does the work.
  • We don't have anyone volunteering to do the bot work yet.
  • The remaining disagreement seems to be about whether we should do more design work and come to PR with more ideas, or whether to go ahead and try to implement just the categories.

If that's accurate, my suggestion would be something like this:

  1. Find a bot volunteer. Nothing can be done without that.
  2. Work on a more detailed design of a minimum implementation of categories, and demonstrate it running on a separate copy of a PR page so that it can be evaluated.
  3. Repropose the idea at PR using that as the model, so that people like Allen3 with real experience can comment with more precision.

I take Geometry Guy's points about a full redesign, but I think a small win, and demonstration that this workshop can achieve something, is worth a lot at this point. Mike Christie (talk) 23:23, 2 November 2007 (UTC)

A small win is definitely needed now. Moving the review to the talk page - although I remain convinced that's the way to go - clearly carries some baggage. So I'd suggest that trying out a categorised version of the PR review page might give a quick win. Or not, but either way, things will have been seen to be done. Not just jaw-jaw. --Malleus Fatuarum 23:33, 2 November 2007 (UTC)
That seems like a good summary to me, and I'm not completely against cashing in on a win idea, but I think we need to build up the bank too. Actually, the prerequisite for categorization of the PR page - automatic archiving - would already make it a big win, not a small one. As Marskell mentioned above, this has been on the PR agenda for some time. Also, it may translate easily to GA: I have raised the idea of automation of GAN and GA already, and the response was generally positive. As for the talk page question, I can see in my mind a "best of both worlds" situation in which reviews are on a stable page, but are transcluded to the article talk page. However moving either GA or PR in this direction means getting the automation working first, so that the technicalities are invisible and do not take up editor time. Geometry guy 00:10, 3 November 2007 (UTC)
OK, it seems that a bot writer is the next step then. I'll draft a note to leave at WP:BOTREQ and put it here for a quick review before posting. Other than the three who've been suggested -- Gimmetrow, Dr pda and Rick Block, the only other bot writer I am familiar with is Daniel Vandersluis, who wrote the GAC page statistics bot. I think Daniel is ramping down his activity, and so perhaps wouldn't be the best candidate. We should post at BOTREQ, and drop a note to the other three as a courtesy since they clearly have the knowledge and capability -- if they are indeed interested and have the time, that would be great. Mike Christie (talk) 00:58, 3 November 2007 (UTC)
Asking one of them first may pan out, and they are familiar with GimmeBot and articlehistory, so can integrate them. SandyGeorgia (Talk) 01:10, 3 November 2007 (UTC)
OK, I posted a note to the three of them. Will draft something for BOTREQ in the next day or so and post it here for review. Mike Christie (talk) 04:04, 3 November 2007 (UTC)
I posted to user Misza13, whose bot is archiving this page, here. S/he said it might be tricky because of the transclusion. Marskell 15:38, 3 November 2007 (UTC)

Good progress

Apologies for the recent silence; it's not a sign of flagging interest (I've been away for a week's holiday). It's great to see the progress so far, and increased input from PR veterans. The mock-up looks good. I had envisaged maybe linking the article categories to the corresponding sections of a separate reviewer-list page, but having this list on the same page is even easier to navigate (providing it doesn't get too crowded). Anyhow, nice work all ;) EyeSereneTALK 17:32, 5 November 2007 (UTC)

There is one thing we can do...

I had said above that the list of volunteers before the categories might be the cart before the horse. But it occurred to me that having a free form list on PR itself might make the page awfully muddled and that a sub-page, prominently displayed in the PR instructions, might work better. Rather than starting a thread about whether we should start the page, I have started a page: Wikipedia:Peer review/volunteers. Per the "let's get a small victory" I think we can run with this. It's a matter of propagation, I think. Ask everyone you know to sign up. When it gets to, say, fifty people I think we can debut it on the main PR. Sound good? Marskell 12:32, 9 November 2007 (UTC)

Yes. I agree. It is an easy and helpful addition to make to the PR page. Geometry guy 18:41, 9 November 2007 (UTC)
Agreed and signed. Will this page also track the numbers of reviews undertaken and/or active/inactive reviewers? (I can't remember what consensus was about this, or even if there was one!) EyeSereneTALK 22:11, 9 November 2007 (UTC)
When it grows we would need to keep track of who is active. That should probably be in the instructions. Whether it's working or not should be clear from the reviews themselves. We can do random audits. Marskell 11:23, 10 November 2007 (UTC)

Moving on with volunteers

While greater minds discuss the bot issues, I'm wondering if it's time to develop new wording for the PR page to incorporate the volunteers list. Marskell (talk) 07:23, 21 November 2007 (UTC)

Yes, I agree. Geometry guy 21:04, 21 November 2007 (UTC)
I think when we get to discussion of direct changes to other pages it's time to move the discussion to the relevant talk pages, so that we can be sure everyone actively involved in the content review process has plenty of time to review the change. I've posted a suggested way to add the volunteer page to the PR instructions in a thread at PR. The particular wording I suggested is just a strawman to get the conversation started. Please comment over there, and with luck we can make the change in a day or two. Mike Christie (talk) 13:20, 23 November 2007 (UTC)
When I started a thread there it received very few comments. I'll try to write up a formatted example tonight. Marskell (talk) 10:28, 25 November 2007 (UTC)
I think the lack of comments also indicates a lack of opposition. I was planning to change the instructions template later today per the note I posted, but if you have a better format I suggest you just boldly implement it and let people modify it if they think it can be improved. Mike Christie (talk) 11:39, 25 November 2007 (UTC)

Comments

The discussion seems to have progressed significantly. I thought I would chime in. There are places where FA and GA work well, and places where they hardly work at all. I have seen similar discussions but I thought I would post these thoughts here. Basically, the place where FA and GA lack is determining the accuracy of the article. They are really fine at detecting bad grammar, MOS violations etc. but when it comes to content, both places, like much of the wiki, sufferes from lack of experts, especially on more obscure topics. My only experience at FAC is with Rock Springs massacre, which was an almost useless endeavour. The article easily passed, probably because I spent so long on it, but an article like that is unlikely to have a leading expert on Wikipedia, heck, comparitively, almost nothing has been published on it, I may indeed be one of the world's leading experts on that topic now. ;) I am not sure how this can be addressed other than by targeted volunteer drives on wiki for processed like PR, GA and FA. IvoShandor (talk) 14:33, 25 November 2007 (UTC)

Glad to see you back from the dead. :-) Ling.Nut (talk) 14:35, 25 November 2007 (UTC)

This topic and the next

I've had a brief exchange with Gimmetrow on our talk pages, and he has said he is willing to go ahead and write the code we will need for the PR bot, but that he is unlikely to get to it for perhaps two months. I think this is great news, since Gimmetrow is clearly the best-qualified person for this.

However, the time delay means we can move on to another topic. So I have some suggestions. The first is that I'd like to nominate Geometry Guy to work directly with Gimmetrow as the representative of this group's goals and intentions on the PR bot. I think it wouldn't be the best use of this page to have all the technical details hammered out here. If Geometry Guy is willing, he is clearly well qualified to work with Gimmetrow, and summarize any questions of detailed functionality so that they can be discussed in this forum.

The work on the bot should proceed hand-in-hand with notification to the appropriate groups -- certainly PR should be notified, at a minimum, though I think all the regulars there already read this page. I don't know if there's anyone else who should be notified so that they can participate.

Second, the time delay till Gimmetrow can work on this implies that we can move on to another topic. Take a look at Wikipedia talk:Content review/workshop/Archive 1#Straw poll for Current Topic, which was the straw poll where we selected PR as the current topic. I suggest we follow that format again here: if you want to disagree with the approach I'm outlining in this post, let's have that discussion in this section; if you want to start talking about the next topic, create a new section called "New Straw Poll" or something similar and we can get started. Mike Christie (talk) 16:15, 25 November 2007 (UTC)

I updated the PR template with the volunteers page, both in the main intro and in the numbered nomination procedures list. Hopefully that will be our small victory. We'll have watch to see if people utilize it.
Before leaving PR, what do you think of Wikipedia:Requests for feedback? Does it really need to exist? Anyway, moving back to a strawpoll makes sense. Marskell (talk) 16:44, 25 November 2007 (UTC)
Thanks for doing that. As for RFF, in the long run, perhaps it does not need to exist: its function could be absorbed into PR. At the moment, though "less developed" articles are explicitly excluded from PR, and the process is primarily used for grooming articles for FAC and GAN. I'd like to see this change, but that takes time. Geometry guy 17:54, 26 November 2007 (UTC)
PS. I'm certainly willing to work with Gimmetrow.

Another bot idea

Here is an idea that may be useful for increasing participation, but would require a bot.

What if every WikiProject had a "Reviewers needed" section on its main page, with subsections for WP:FAC (and WP:FLC), WP:GAC, and WP:PR, and others if needed? The bot would go through the nominees at each (FAC, FLC, GAC, PR) page, check the talk page of each nominated article for WikiProject tags, then list the articles in the appropriate section of each WikiProject's page.

I assume the bot would update the lists every few days, and would both add new requests and remove old ones (that were no longer on FAC, FLC, GAC or PR). The assumption is that the notices would attract more reviewers - I already do this for my FAC articles (though on the WikiProject's talk page).

I like the idea and figure the details would need to be worked out, but wanted to see what others thought here, Ruhrfisch ><>°° 02:48, 26 November 2007 (UTC)

At WikiProject Mathematics, there is a subpage called Wikipedia:WikiProject Mathematics/Current activity, which is mainly autogenerated by Jitse's bot. However, I'm not convinced that very many people check it, and the WikiProject Mathematics page itself isn't particularly active. If you want to attract the attention of a bunch of mathematicians, the place to post is on the project talk page, and there, a personal message from a nominator is more likely to get a response than something automatic. I imagine other WikiProjects are similar. Geometry guy 17:45, 26 November 2007 (UTC)
It does depend on active Wikiprojects to work - I know the ones I am involved in are decent for feedback on specific questions, less so on requests (on the talk page) for PR or GAC or FAC feedback. Oh well, thanks for the feedback, Ruhrfisch ><>°° 21:16, 26 November 2007 (UTC)

A-class

A-class should be abolished and all current articles rated as A-class should be regraded as GA-class because:

  1. A-class is the least used and hence least useful of the gradings, and the natural candidate for elimination in any rationalisation.
  2. GA and A-class standards are now little different, making a division between them unnecessary.
  3. The sole goal of A-class review, the potential review and improvement of an article, is duplicated by alternative processes (WP:FAC, WP:GAN, WP:PR).
  4. The presence of a fourth mechanism for review and improvement splits effort across processes, making each individual process less rigorous than it could be.
  5. At present, A-class status is not processed uniformally across projects, with multiple projects having different mechanisms for bestowing A-class. Existing WikiProject A-class review processes could be merged with GA review, thus delegating some of the GA reviews, increasing the number of GA reviewers, decreasing both the GA and A-class review backlogs, and unifying the separate A-class procedures.
  6. Removal of A-class re-harmonizes distinct assessment processes (FA and GA; v1.0; WikiProjects) into a more cohesive cross-project structure of FA, GA, B, Start, Stub.
A Step Further?

B-class could be rebranded with a new name, e.g. "Standard-class", which would be more descriptive and less judgemental, as well as being an amusing alliteration (standard-start-stub). DrKiernan 07:56, 3 December 2007 (UTC)

I agree. When we did the poll, analyzing the terminology of the various categories enjoyed some support. Shall we get back to it? Marskell 11:01, 3 December 2007 (UTC)
Is this a proposal for the next topic to be discussed? I can't see a new straw poll section anywhere, but I'd be happy to run with this one in the absence of any other suggestions... EyeSereneTALK 19:08, 3 December 2007 (UTC)
I would also agree with the abolition of the A-class. With the raising of the bar and toughening up of the criteria for GA status it doesn't seem like it's any longer necessary. --Malleus Fatuorum 20:48, 3 December 2007 (UTC)
As was recently explained to me, A-class was intended to be a v1.0-specific tag to indicate that an article was not only well-written etc, but content-checked by subject specialists in a way that is not currently done at, for example, GA - this is why it is awarded by WikiProjects where such experts can be found. Aside from the potential disruption to the v1.0 process, are we suggesting that expert review should now come under one of the other processes? EyeSereneTALK 20:57, 3 December 2007 (UTC)
I don't find that explanation to be very convincing. Many articles go straight to FA without ever going through A, or even GA, and by implication are never reviewed by subject area specialists. IMO, A-class as a subject area review only makes sense if it's a mandatory step on the way to FA. Which it isn't. --Malleus Fatuorum 21:12, 3 December 2007 (UTC)
One further point: anyone can join any wikiproject they like. Being a member of a project does not guarantee subject area knowledge. --Malleus Fatuorum 21:14, 3 December 2007 (UTC)
Perhaps I should clarify: A-class, although it is listed as part of the quality scale, is apparently (or at least was intended to be) something of a sidestep on the review ladder for the v1.0 release (see Wikipedia:Version 1.0 Editorial Team/Assessment). This is borne out by the fact that an article can be both GA and A-class at the same time. The impression I get though is that this is not understood by most editors (it was news to me!), which would imply that it either doesn't work, or is not being applied correctly, or has evolved into something else, or means different things in different places, or... In any case, I do think we need to be discussing this with members of the v1.0 assessment team before we decide to bring their house down ;)
Whatever the issue with A-class, I agree that many - if not most - articles (even FAs) are never really content-checked; instead we rely on knowledgeable readers to fill in gaps over time. Personally I don't believe that this is an area we can proscribe for, since there are never going to be enough experts in all areas to meet the need, but even so it ought to be part of the quality process somewhere. EyeSereneTALK 21:57, 3 December 2007 (UTC)
I have no desire to even blow slightly on the v1.0 assessment team's house, never mind bring it down. :) But I do wonder about this perceived lack of subject area review. I don't know what the figures are, but I would imagine that most articles are now under the remit of at least one wikiproject, which presumably would contain at least some subject area experts. As the article was being developed it would have been looked at frequently, and the requirement for verification by reliable sources, inline citations and the like must mean that by the time it hits FA there really ought not to be too much wrong with it content-wise. And processes like FA and GA are, I believe, quite good enough to smell out a lack of balance in an article. So I'm afreaid that I still don't see the purpose of A-class. --Malleus Fatuorum 22:29, 3 December 2007 (UTC)

In principle it is a worthwhile topic to discuss the relation of of the Start-Stub-B-A system to good articles and featured articles, but from this starting point the discussion may simply become divisive. I urge people to start topic discussions by proposing a topic, not a solution. Mike's obviously busy right now, but we can still try to keep to the spirit of the workshop that he facilitates.

For reference, the previous straw poll is here, and this workshop has already touched upon some of the issues here. The most recent proposal to abolish A-Class was made barely a month ago at WikiProject Council. It didn't fly for the basic reason that Start-Stub-B-A are individual WikiProject assessments, whereas good article and featured article are project-wide. The scheme is well-developed at many WikiProjects, and asking each of them to abandon it is a bit like asking the good articles and featured articles projects to merge. It won't happen.

Note contrary to the above, it is not a v1.0 assessement, but a WikiProject assessment. However, individual WikiProjects mostly follow the 1.0 scheme, which means that good articles are either GA-Class or A-Class, while featured articles are automatically FA-Class. In my opinion, this was an historical mistake, and the Start-Stub-B-A system should be, at the very least, orthogonal to the good article process. See Talk:John von Neumann for an example of the independence of WikiProject assessments. I disagree with almost all of the numbered points above, and am happy to repeat all the arguments if necessary. In the meantime, I would like to politely request that we don't cloud the issue further by using GA-Class and FA-Class as synonyms for good article and featured article. They are logically distinct, as the classes are assigned by WikiProjects; furthermore good articles can be A-Class instead of GA-Class, and some projects do not give GA-Class to good articles which are weak in content from the project perspective. Geometry guy 23:32, 3 December 2007 (UTC)

Geometry Guy, thanks for jumping in when I was slow to respond. I agree with the points you make above; if we do want to revisit the hierarchy we can certainly do so, but I think it would be best to have that emerge from a consensus after we revisit the possible topics. I'll start a separate section to talk about that in a moment (once I finish helping my seventh-grader with her maths homework). Mike Christie (talk) 00:26, 4 December 2007 (UTC)
Thanks for the correction G'guy ;) EyeSereneTALK 09:47, 4 December 2007 (UTC)
My opinion remains unchanged. DrKiernan 09:52, 4 December 2007 (UTC)
  • "B-class could be rebranded with a new name, e.g. "Standard-class", which would be more descriptive and less judgemental, as well as being an amusing alliteration (standard-start-stub). DrKiernan 07:56, 3 December 2007"
As a fresh invitee here, before contributing anything useful or even reading the whole page, I must allow my otherwise-useless knack for alliteration to extend DrKiernan's idea. The complete WP ranking system then becomes:
  1. Stub
  2. Start
  3. Standard
  4. Sterling
  5. Starring

Perhaps I can contribute something more useful as time permits. Cheers, Unimaginative Username (talk) 02:37, 5 December 2007 (UTC)

Draft for BOTREQ

Here's a draft note for the bot request page.

Peer review automation bot

I'd like to request a bot writer to work with the Wikipedia talk:Content review/workshop on a suggested change to Peer Review. There has been quite a bit of talk at the workshop about possible changes to PR and the current plan is to implement categorization on it. Because the page is currently manually archived, this would require a bot -- the manual process is already tedious and would become unmanageable without automation if categorization were introduced. We'd also like to change the page to list only links, rather than the whole existing review. Here's a mock-up of how the peer review page might look: Wikipedia talk:Content review/workshop/Peer Review mockup.

We are hopeful that this is the first step in a reinvigoration of Peer Review. If this works, we hope to come up with more ideas to help improve not only peer review but other content review processes. However, this first step is in some ways the most important: peer review is a key part of Wikipedia, and it's not working as well as it could. We believe organizing the page will really help participation, and we'd like to work with someone who would be interested in continuing to participate and who can help us improve our ideas and make them implementable.

If you're interested, post a note either here or at the workshop page, and we can talk about implementation details. Thanks. Mike Christie (talk) 02:13, 4 November 2007 (UTC)

Comments here

Let me know what you think. If you want to edit above, I suggest using strikeouts for deleted text and italics for added text so that others here can see what the changes are.

Note that the workshop link in the paragraph above doesn't work because we're on this page; it would work in the botreq page. Mike Christie (talk) 02:13, 4 November 2007 (UTC)

Seems fine to me. I shortened one sentence but it's very minor. Marskell 15:09, 4 November 2007 (UTC)
It may be worth noting that there is already some automated archiving and processing of the peer review page by GimmeBot, so input from Gimmetrow will be invaluable whether or not a different user responds to the bot request. Geometry guy 15:28, 4 November 2007 (UTC)
I think I'll leave that till someone shows interest; I am pretty sure Gimmetrow is too busy to do this, and Gimmebot's involvement is just part of the technical details that will need to be discussed. Mike Christie (talk) 19:18, 4 November 2007 (UTC)

OK, I just posted the request. Let's see if anyone is interested. Mike Christie (talk) 19:24, 4 November 2007 (UTC)

I think it would be a good start if you could give a description of what the bot should do. The more details, the better. It would give a bot operator a better idea of what you'd like done. --Erwin85 22:10, 4 November 2007 (UTC)
We would like:
  1. The PR summary page to be generated by the bot, in the format given in the mockup.
  2. PRs that have aged past a certain point to be archived by the bot, meaning removed from display on the page.
  3. The ArticleHistory template on the article talk page to be updated by the bot.
Per discussion on this page, a consensus is that the best way to do this is to change the PR request method so that a user simply adds a template to the article's talk page. The bot would find these templates and generate the PR page from them. The age of the generated PR subpage would then be the way the bot knew how to remove the PR from the list, when it aged past the limit (say 30 days).
I would think the bot operation would look something like this, though others here are better at bot operations than I am and may correct this:
  • Scan for all PR-Request templates.
Generate PR page header
For each article
if it's older than the max age, update ArticleHistory to show it has a completed PR; remove the PR-Request template from the page.
else generate a listing for the article (in the right category for that article; the cat is a param on the PR-request template)
Generate PR page footer
Improperly filled out templates would cause an exception report subpage to be written
Does that clarify things? Should I post this at the BOTREQ page? Mike Christie (talk) 22:27, 4 November 2007 (UTC)
No, you should not. Your description combines two separate bot activities: archiving, and autogeneration of the review page. I support both of these activities, but they are different. The consensus above supports the idea of categorizing the review page. The obstruction is that this adds to the archiving overhead. The solution is that the archiving should be automated in the same way as archiving of FACs is automated. However, automated generation of the peer review page (which I support) needs further discussion.
The archiving task is not that complicated, and should be a small modification of the code used by GimmeBot to archive FACs. There are three situations in which an archiving bot could act: first, when a current peer review is older than a month; second, when an active peer review has not received a new comment in two weeks; third when an editor replaces the peerreview template by an oldpeerreview template. The first two of these are currently largely handled by Allen3 and GimmeBot: it would not be hard to automate them entirely. The third case requires the bot to detect the template change, and may not be so easy.
What then does the bot do if it has to archive? Well, first, it must move [[Wikipedia:Peer review/ArticleName]] to [[Wikipedia:Peer review/ArticleName/archiven]], where n is the next free archive. Second it must remove the listing of the review from Wikipedia:Peer review and add it to the archived review list at Wikipedia:Peer review/Some monthly archive page. Third, it must update the ArticleHistory template of the article to add the peer review event, with a link to the archived page. This is more-or-less what GimmeBot does for FACs, and I expect the code could be easily modified. Geometry guy 23:43, 4 November 2007 (UTC)
I was afraid that the hiatus in my involvement had led me to misunderstand the current proposal; thanks for the clarification. Could you post an accurate version under my request at WP:BOTREQ? Thanks. Mike Christie (talk) 23:46, 4 November 2007 (UTC)
The bot request is fine. I don't have time to clarify the technical details right now, but will do so tomorrow evening (UTC) or Tuesday if necessary. Geometry guy 23:54, 4 November 2007 (UTC)
I haven't been editing a lot either and I very much hope we don't accidentally let this deflate. The main thing re above is that we should not make a request for automatic page creation, as G'Guy mentions. Archived after 30 days or two weeks with no comment and leave it at that, for now. Marskell 10:43, 6 November 2007 (UTC)
I suspect that if we place a more detailed specification on WP:BOTREQ it would make it easier for a bot operator to decide to help write the bot for us. As it is the idea (and this workshop) may languish for lack of an implementer.
I also noticed this thread at the Village Pump, about sorting peer reviews. Perhaps commenting there to direct people to here and to BOTREQ would be useful.
FYI, I'm starting a two week business trip this afternoon. Most of the hotels will have net access, but I might be offline for as long as 36 hours once or twice, so I may be a bit slow to respond sometimes. Mike Christie (talk) 12:29, 6 November 2007 (UTC)

Automation of peer review

There are several ways that this could be done. In the following approach, I have tried to separate out three independent tasks. Only the first task is actually essential, but it is also the most complicated.

Archiving bot specification

This specification is essentially the task that GimmeBot performs at WP:FAC. Probably a small modification of the code written by Gimmetrow (if he is willing to share it) would provide this functionality.

The bot should run once per day. It should first query "What links here?" to a specific template, say {{archive peerreview}}. The talk pages on this list should then be used to compile a list of article names. For each article name AN in this list the bot should perform the following tasks.

  1. Remove the item [[Wikipedia:Peer review/AN]] from Wikipedia:Peer review.[1]
  2. Move the page [[Wikipedia:Peer review/AN]] to [[Wikipedia:Peer review/AN/archiven]], where n is the smallest positive integer greater than the integers for which [[Wikipedia:Peer review/AN/archiven]] already exists.
  3. Add a transclusion {{Wikipedia:Peer review/AN/archiven}} to the top of the current archive [[Wikipedia:Peer review/Current month]].[2]
  4. Remove {{archive peerreview}} from [[Talk:AN]].
  5. If [[Talk:AN]] does not transclude {{ArticleHistory}}, add {{oldpeerreview}} to [[Talk:AN]].
  6. If [[Talk:AN]] does transclude {{ArticleHistory}}, then add an event to the article history with actionX=PR, actionXoldid=the oldid of the current article, actionXlink=[[Wikipedia:Peer review/AN/archiven]], actionXdate=the date of the current oldid, actionXresult=reviewed. (Here X is the smallest positive integer which is greater than the event numbers of current ArticleHistory events.)

Footnotes.

  1. If the peer review page entries are autogenerated (see below), then this step will not be required.
  2. This follows the current practice at WP:PR. However, in most archives, the oldest entries are at the bottom, not the top. Also, the current practice of using archive names of the form [[Wikipedia:Peer review/Current month]] is different from WP:FAC, which uses archive names of the form [[Wikipedia:Featured article candidates/Archive/Current month]]. The FAC practice is more robust: in the unlikely event that someone nominated, say, October 2007 for Peer review, there would be problems!

Automatic archiving

This is optional. With the above bot specification, any editor can archive a peer review request by replacing {{peerreview}} with {{archive peerreview}} on the article talk page. However, some archiving could be done automatically.

Again, this runs once per day. The bot queries "What links here?" to {{peerreview}} to produce a list of talk pages, hence articles under peer review. For each such article name AN, the bot queries the edit history of [[Wikipedia:Peer review/AN]]. This allows it to determine the date that the peer review started (either the creation of the page, or the date that a redirect to an archive was replaced)[1] and the date of the last edit.

  • If the last edit is more than x days ago, the bot replaces {{peerreview}} by {{archive peerreview}} on [[Talk:AN]].
  • If the article has been under review for more than y days, and there have been no edits for z days, the bot replaces {{peerreview}} by {{archive peerreview}} on [[Talk:AN]].

Current practice would suggest x=14, y=30, and z small (say 2).

Footnotes

  1. It may be easier for the bot to maintain a record of when the peer review started.

Automatic generation of the peer review list

This is again optional, and is essentially independent of the first two tasks. However, it is such a simple and general idea, that I wonder why it does not exist already.

The role of this bot is to compile a list of articles in a category. Like a talk-page archiving bot, categories can request to be compiled by transcluding a template with a name like {{list category}}. The bot runs regularly, perhaps once or twice a day, or perhaps different frequencies can be requested in the template. (Care must be taken to ensure that requests involve categories of reasonable size and updates of reasonable frequency.) For each such category [[Category:CN]], the bot maintains a list of articles in the category, together with a date. If a new article is added to the category, the bot adds it to the list, together with the current date. If an article is removed from the category, the bot removes it from the list. In other words, the bot has a list of the current articles in the category together with the date when they were most recently added.

On each run, the bot uses this record to write a list of dated articles to a subpage [[User:Name of bot/CN]]. The most flexible way to do this is to use a template associated to category CN, with a name like {{category CN format}}. The subpage would then be a list of entries of the form {{category CN format|name=ArticleName|date=DateAdded}}. There should be options for the bot to list these entries alphabetically or by date (ascending or descending).

Why is this useful for Peer review? Well there would be one category for each of the 10 topics: [[Category:Topic peer review requests]] and {{peerreview|topic=topic}} would place the talk page of the article in the corresponding topic category. The templates {{category topic peer review requests format}} would all be the same: they would format the article names as they would appear on the peer review page. The peer review page would then transclude each of the [[User:Name of bot/Topic peer review requests]] subpages. This is better than a bot directly generating the peer review page, because it separates form and content: the peer review page and templates determine the format, the bot only determines which articles are listed.

Note. Unlike the other two activities, this is a "simple bot" according to my definition: it never reads a page, but only reads "What links here?" and only writes to a small number of subpages. This means it places very little load on the servers and can be run quite often.

Comments

This is not the only way to proceed. In particular, if all archiving of peer review is automated, then tasks 1 and 2 could be combined, and the template {{archive peerreview}} would not be needed. Task 3 is not needed, but would have wider applicability, for example, to GA. Geometry guy 21:57, 6 November 2007 (UTC)

That's it, I think. So where are we at? And what about you G'Guy? Can you program? This is frustrating. Surely we can find the needed Wikipedian. Marskell 08:26, 8 November 2007 (UTC)
Perhaps a next step would be to broaden the publicity we're trying to get. We've posted at the bot request page and peer review, as well as here; the link DrK posted at the Village Pump did not give any details. If what we're doing has merit, would it be better to post more details at the Pump in order to try to get some more interest? And perhaps bot writers would have been more likely to respond if we had given a more algorithmic statement of the task at BOTREQ -- bot writers may be more interested in assessing whether they can do a task than in following a link and reading a discussion. That might mean it's worth reposting the request in a more technical format.
Alternatively, it might be the case that there is genuinely not enough interest in fixing these content review processes to attract a bot-writer. If we can't fix that by more PR, we'll just have to have patience until someone with the right skills and interests comes along. We might speed that up by, for example, proactively asking Gimmetrow if he would be willing to release his archiving bot code. Mike Christie (talk) 08:41, 8 November 2007 (UTC)
Yes, go back to the pump. Both Proposals and Technical, perhaps. Gimme may be our only man but I don't want to throw extra work in his direction until we've exhausted a search for someone else. Marskell 08:50, 8 November 2007 (UTC)
Although I can program in general, I don't have experience with this kind of work. I can think of a couple of editors who might be willing to help, particularly with the third task and will ask. I agree with the ideas to advertise, and also to ask Gimmetrow if he is willing to provide a link to his source code. I also think we should update the workshop front page and the bot request in the light of where we are now. I also think we should consider what should be the next question. Geometry guy 18:08, 8 November 2007 (UTC)
Concerning your last point, it looks like collaboration on this project has dropped right off. I'm wonder how many are, like me, just observing with interest whilst the technical stuff is discussed... and waiting for something to come up that we feel we can contribute to. In any case, we could be in limbo indefinitely if we put everything on hold until we find a bot writer; maybe it is time to move things along? EyeSereneTALK 20:28, 9 November 2007 (UTC)
Well, the thread two down is an attempt to move along while we wait for a bot writer. Sign up as a PR volunteer EyeSerene! Marskell 20:32, 9 November 2007 (UTC)
...and put my money where my mouth is, eh? Fair enough ;) I've added myself to the new page using the link below, although I'm pretty busy here at the moment with an interesting pre-FAC copyedit. Regarding copyediting, why is that a separate category on the Peer review/volunteers page? EyeSereneTALK 22:07, 9 November 2007 (UTC)
I was thinking of all-purpose copyeditors, such as Tony1. Marskell 11:22, 10 November 2007 (UTC)

Update

Thanks to the help of Carl(CBM) and his VeblenBot, the third task above, i.e., automatic generation of the peer review page, can now be done. An automatically generated list can be found at User:Geometry guy/Peer review. This exercise has further confirmed my belief that mundane, complex, error-prone activity should not be done by human beings. At the moment requesting a peer review involves:

  1. Adding {{Peer review}} to the talk page.
  2. Creating Wikipedia:Peer review/Article Name starting with a level 3 heading of the form ===Article Name===
  3. Transcluding this subpage onto Wikipedia:Peer review.

The first task puts the article in Category:Requests for peer review, and VeblenBot provides a list of dated entries. My demonstration uses this to automate the third task. It reveals examples where steps 2 and 3, or just step 3, have been omitted. Also there were many examples where the heading in step 2 was malformed (e.g. had a space before it, or no new line after it, or was the wrong level, the wrong title, no link, etc.). I have tried to fix these errors.

Note that VeblenBot does not track subcategories like Category:Requests for Biography peer review, some of which appear to be listed on the current PR page. Geometry guy 08:49, 12 November 2007 (UTC)

The Biography peer reviews could be added to the tracked category by adding Category:Requests for peer review to the WP Biography template code at Template:WPBiography. DrKiernan 11:34, 12 November 2007 (UTC)
Yes that is certainly one way to proceed. Alternatively, VeblenBot is a flexible guy and can list any category: we just ask Carl(CBM) to add the category name to VeblenBot's list of categories. However, this would mean that articles in different categories would be on different lists.
Do you know which categories are listed at WP:PR? I don't think all of the subcats of Category:Requests for peer review are there, are they? Geometry guy 18:27, 12 November 2007 (UTC)
I routinely tranclude the biography peer reviews by hand, but peer reviews from other projects are only listed if the nominator adds the transclusion. I guess the projects work out their own peer review mechanisms. DrKiernan 12:56, 13 November 2007 (UTC)
It sounds to me that the best way to reflect current practice is to do as you suggest, and modify Template:WPBiography to add articles to the general PR category. Many thanks also for tidying up the old peer review requests detected by VeblenBot: the tiny bit that I did made me realise just how much work is involved! I notice that you also updated User:VeblenBot/C/Requests for peer review so that the demonstration page looks better. Thanks for that too. Since this idea appears to work, I will now ask Carl to switch on automatic updating of VeblenBot's subpage, after which further manual edits should not be made, as they will be erased by the bot. Geometry guy 18:53, 13 November 2007 (UTC)
I've now modified Template:WPBiography to add Requests for Biography peer review to the main peer review category. Geometry guy 12:15, 17 November 2007 (UTC)
Okay, the problem with this is that peer reviews for biography are on subpages of a different page, so for this to work, redirects need to be provided from the peer review subpage to the biography peer review subpage. That sounds like extra work to me (hence a bad idea). I notice that several other projects list their peer reviews on the main page. I think it would be better if they were in separate lists. Geometry guy 11:11, 19 November 2007 (UTC)

(unindent) So, how do people what to handle the wikiproject peer reviews? Complete separation? Complete integration? Ignore them? They sort themselves out? Separate lists on the peer review page? DrKiernan (talk) 18:46, 19 November 2007 (UTC)

I would say separate lists on the peer review page for those projects/peer review processes that wish to be involved. If we categorize peer reviews into 10 topics, then there could be internal cross-links from the topics to particular peer review processes, e.g., from Natural Sciences to Scientific Peer Review. Geometry guy 20:32, 20 November 2007 (UTC)

Auto-generating peer review page

VeblenBot is now automatically updating the category list. The demonstration page is pretty close to the current PR page, even though it does not include the 18 current bio PR requests. There is some difference in ordering, because VeblenBot lists articles by the date in which they were added to the category, and the current PR page is only an approximation to this. Geometry guy 20:44, 13 November 2007 (UTC)

To be more precise (the original formulation of my above message was wrong), VeblenBot checks the category contents at 20 minutes past the hour every hour, and updates the subpage if there has been a change. I think this is fantastic, and could solve many other problems. Geometry guy 23:29, 13 November 2007 (UTC)

I'm an idiot: I just messed up by making a mistake in editing the peer review template. The template is fixed now, but the peer review requests were momentarily taken out of the category, so the bot now considers that they were all requested very recently. The moment was a matter of minutes, so the software is pretty efficient! This is no way an excuse, but it is hard making edits to category outputs of templates, because there is no way to preview the results... sigh, blush... Geometry guy 00:33, 14 November 2007 (UTC)

Perhaps the bot updates its info too frequently... --Ling.Nut 01:31, 14 November 2007 (UTC)
No, that's not a problem: the bot simply reflects what is in the category, and in fact, I had fixed my error and restored the category well before the update. The problem was that the Mediawiki software was too fast for me: it had depopulated the category before I fixed my mistake, and then had to repopulate it, with new date-stamps for the re-added articles. When the VeblenBot next asked the toolserver for new information, the software cheerfully reported that there were a lot of recently added items in the category. I could revert User:VeblenBot/C/Requests for peer review back to the previous version, and VeblenBot would not change it until the next change in the contents of the category. {{WPBiography}} is edit protected: we might consider doing the same with {{peerreview}}, or at least putting a comment in the code to warn editors. Geometry guy 09:11, 14 November 2007 (UTC)
(edit conflict) I'm in the middle of a business trip and have only had enough WP time to keep up on a couple of FACs that are ongoing, so I haven't contributed here for five or six days. I'm impressed by the progress so far. Just to make sure I understand what the current situation is, I'd like to recap and see if I have it right.
Currently VeblenBot's subpage is constructed by having the bot look for all articles with the {{peerreview}} template, and building a page that lists all these articles by using Template:CF/Requests for peer review. This template has three parameters: name of article, a number which is presumably a category apparently not being used yet, and a date. The name of the article is used to transclude the actual peer review. The date is simply output to the page text. The subpage to which all this is written is User:VeblenBot/C/Requests for peer review.
No progress has yet been made on either of the two archiving tasks which we'd discussed above. The date output on VeblenBot's subpage could be used for the auto-archiving task. There has also been no progress towards categorization except for the creation of the category argument to the Template:CF/Requests for peer review template. To populate this, a change would have to be made to Template:peerreview to give it a category argument. Then VeblenBot would have to sort by that category, and also create the cat headings on the subpage. Users could then use the category if they wished.
If all that is correct (and I suspect I've missed something), then we still need to address the error Geometry Guy made, which is rather easy to make and would lead to VeblenBot restarting peer review aging. Perhaps a way to temporarily disable VeblenBot while changes to the peer review template are being made? (Edit conflict: I see Geometry Guy is already commenting on this.)
One other question. What remains to be done before this could be used to populate the actual peer review page? Does this method of population conflict with the way Allen3 archives the page?

-- Mike Christie (talk) 09:21, 14 November 2007 (UTC)

Re:Allen3. I don't think so, even if someone manually archives (as I did with the old peer reviews) the archived peer review is removed from the category and when Veblenbot updates there's no change to the page. DrKiernan 09:43, 14 November 2007 (UTC)
Yes that is all pretty much correct.
  • Minor points. VeblenBot doesn't check for the template directly, but looks at the category: hence other templates (such as {{WPBiography}}) could put articles in the same category. Also, the second parameter in {{CF/Requests for peer review}} is just the namespace number: this parameter will not be used for anything — all the pages listed are talk pages (Talk is namespace 1).
  • Categorizing. As Mike points out, we need to modify {{peerreview}} to accept a topic name, but the mechanism is not the one imagined, because the bot doesn't read the template directly. Instead {{peerreview|topic=Arts}} would put the talk page in [[Category:Arts peer review requests]] (say) and VeblenBot would diligently update subpages of the form [[User:VeblenBot/C/Arts peer review requests]]. The PR page would then transclude each of these under suitable headings.
  • Archiving. At the moment, Allen3 and others (including DrKiernan) archive the peer review of Foo in three steps:
    1. Replace {{peerreview}} with {{oldpeerreview}} on Talk:Foo (or do some {{ArticleHistory}} stuff);
    2. Remove the transclusion {{Wikipedia:Peer review/Foo}} from Wikipedia:Peer review;
    3. Add the transclusion {{Wikipedia:Peer review/Foo}} to [[Wikipedia:Peer review/CurrentArchive]].
The bot makes step 2 unnecessary, as long as step 1 is carried out: the bot will do step 2 automatically within an hour. As DrKiernan points out, doing step 2 manually is harmless, but the bot will undo step 2 if step 1 is not carried out. If desired, VeblenBot could also autogenerate the archives, at the expense of adding a "month" parameter to {{oldpeerreview}}. Also, as I have explained before, I believe this archiving procedure is flawed, as it does not provide a permanent archive link: archiving should also involve a page move of [[Wikipedia:Peer review/Foo]] to [[Wikipedia:Peer review/Foo/archiven]]. At present this flaw is partially fixed by GimmeBot.
  • The error. In some ways I'm glad I made this mistake, as it is better to discover this issue now than later! Disabling VeblenBot would not help: we would have to disable the Mediawiki software instead! I have proposed a fix, anyway (a strongly worded comment in the template source, perhaps reinforced by edit protection).
  • What needs to be done. In other ways, I'm not glad I made this mistake, because we now have to wait a month or so for the automatic page to list peer reviews in the right order again. In the meanwhile we can see if we can provide a demonstration of an automatically generated and categorized page. This should be very easy.
I hope at least that we have some sort of proof of concept. The hard part is the automated archiving. Geometry guy 18:45, 14 November 2007 (UTC)
Why would the process need a "month" parameter in the template? Couldn't the bot just add anything it closes to the current month's archive? The error is a problem with using category timestamps, but there are other ways to figure out order. For instance, the bot could remember the last-read contents of the category, and compare that with its next sweep. This process would avoid most errors from working with the template and template vandalism, and would identify not only new PRs but PRs removed manually. Another way would be to check the talk pages for the time the peerreview template was added, though this would add a bunch of mostly-useless page reads. Gimmetrow 01:10, 17 November 2007 (UTC)
I agree, it is much better to use an archiving bot to add closed discussions to the current month's archive: I was only mentioning in passing that it would be possible to do it via categories (using e.g. VeblenBot). The solutions you suggest to the category timestamp problem are certainly options. A "last-read" category memory would have avoided this error, but only because I was fast enough to fix it before VeblenBot's next update. VeblenBot's method has the advantage that it is server non-intensive, and so can be run very frequently. Therefore it reflects the category contents most of the time. A more sophisticated category memory system may be possible (e.g., storing also the contents 24 hours ago), but it requires someone to write the code :) Geometry guy 11:41, 17 November 2007 (UTC)
Trying to follow some of this technical stuff makes my head a little swimmy. Can we appoint G'Guy and Gimmetrow the bot people and they could work out the model themselves?
In other news, the volunteer list has filled up nicely. Perhaps ready for a debut. Marskell (talk) 11:44, 17 November 2007 (UTC)
That's good news. Sorry that this page has become so technical, but the processes and the technology are interrelated: for example, it would be a bad idea to develop a system which Dr. Kiernan and Allen3 are not happy with. I hope we'll sort it out soon and the discussions can focus more on content review. When Mike is back from his trip, I imagine he will have more time to take a lead. Geometry guy 12:15, 17 November 2007 (UTC)

I get back home on Tuesday, and after a day or so of jetlag I should be more or less back to normal activity. I'm happy to help out by facilitating or suggesting directions for the conversation.

From what I understand of the current situation, perhaps we just need to be patient while the more technically-minded people try to figure out if there's a way to achieve what we would like to achieve. We don't have a commitment from a bot-writer yet, though Gimmetrow's expressed some possible interest. If that comes to fruition or another volunteer comes along we will be kept busy validating and helping to improve the results. (If we don't get a bot-writer interested, we'll have to do something about that, of course.) That's likely to be plenty to talk about. If we jump onto something else while we wait, we'll use up our psychic energy on that. Holding down the level of effort required for this workshop will help make it easier for people to participate longterm.

So for now I'd suggest we don't start any significantly new threads, though I think publicizing the volunteer list is a great idea -- maybe the Signpost, and/or the village pump, would be good places. Mike Christie (talk) 18:12, 18 November 2007 (UTC)

Yes, I agree some patience is needed here, and we can certainly publicize the volunteer list in the meanwhile. I think there is a danger, though, of people losing interest in the workshop if it is not reasonably active. There have been times when it was too active; now it is a bit too quiet, in my view. It is hard to get the balance right! Geometry guy 11:26, 19 November 2007 (UTC)

What happened to Vbot here: [2]? DrKiernan 12:01, 3 December 2007 (UTC)

Many thanks for drawing attention to this. It occured once before, but I thought it was a teething problem. I will ask Carl what is up. Geometry guy 23:37, 3 December 2007 (UTC)

Another update (and good news)

Further to the above discussion, Carl has now fixed the bug, and has also, at my request, made the VeblenBot's category listing more robust by keeping a record of the timestamps of articles in the category. This means that if an article is removed from the peer review category for a short time, and then re-added, the timestamp remains unchanged, and so the position of the article in the peer review list is unchanged. In particular, mistakes, such as the one I made previously with the peer review template, will no longer cause problems. Also, this prevents editors from moving their article to the top of the list by removing it briefly from the peer review category. Not that anyone would try that ;-)

The auto-generated peer review page is now strikingly similar to the hand-made version. There are one or two minor differences in ordering (due to delays in adding listed articles to the category), but the only substantial difference is that a WikiProject highways article is listed on the PR page, but not on the automated mock-up. Actually, the highways review is closed, so this should probably be archived anyway. Anyway, my feeling is that we could try rolling out this technology soon. I'll start a thread on the PR talk page in a day or two, but any comments here are welcome. Geometry guy 20:19, 8 December 2007 (UTC)

Dropping in

I see the PR stuff is making progress.

With a free weekend I could probably get archiving of all PRs set up.

However, I'm not sure archiving to a /archive subpage is the best way to do things. It's the way FAC, PR, and FAR have done things for a while, but when the page moves it's necessary to update at least some of the whatlinkshere. Updating them all seems a little much. Rather than archiving, it is possible, with some fancy template coding, for the peerreview or fac template to pick an available unused page. The page is fixed and never needs to be moved. Since this makes archiving unnecessary, it doesn't need a bot to maintain.

The disadvantage is that it would be a big departure from the way PR and FAC have been done, and potentially a culture shock. It would be closer to how AFD works. Also, the ways I've thought to implement this would appear to require subst'ed templates, or the PR template would keep pointing to a new, nonexistent page.

Another thing to think about. Gimmetrow 02:15, 16 November 2007 (UTC)

I don't follow. DrKiernan 08:20, 16 November 2007 (UTC)
I believe that Gimmetrow is referring to the path name used for transclusion files and asking about changing the name used from the current Wikipedia:Peer review/{{ARTICLENAME}} to something of the form Wikipedia:Peer review/{{ARTICLENAME}}.{{UNIQUE ID}}. --Allen3 talk 13:17, 16 November 2007 (UTC)
Gimmetrow, what are the reasons for updating the whatlinkshere links? I can see there's a minor inconvenience for anyone following a link in a discussion expecting to find a current peer review discussion -- but there is no current peer review discussion once it's archived, so is that really a problem? Mike Christie (talk) 14:13, 16 November 2007 (UTC)
Let's say an article is listed at FAC while a PR is running, and the PR link is noted. The link goes to Wikipedia:Peer review/Article. When the PR is archived, it becomes Wikipedia:Peer review/Article/archive1. The link on the FAC page should be updated too. Otherwise, I suppose most of this issue relates to the big list of old PRs which transclude everything for a month. When anything on that list is archived, the link should be updated.
It would be nice to avoid updating links. I'm thinking it might be easier if the peer review template looked for an empty page of the form Wikipedia:Peer review/Article/reviewN. If /review1 exists, use /review2. Then the page is fixed from the moment the review starts. It can be linked or transcluded from other pages without need for future updates. Gimmetrow 17:04, 16 November 2007 (UTC)
Ah, well I don't think that's necessary. We don't routinely move old peer reviews to Wikipedia:Peer review/Article/archive1. They stay at Wikipedia:Peer review/Article until someone manually moves it, when they want a second peer review. Can't we keep that system with the onus on the editor who requests a second peer review to update the links? -- DrKiernan (talk) 17:11, 16 November 2007 (UTC)
  • That* is part of the problem (that it's not done routinely). The "someone" who later has to fix it is usually ... Gimmetrow or SandyGeorgia ... at least for FACs and GAs. It should happen automatically, and be correct from the get-go. As in FAR notifications, "onus on the editor" rarely works. FAC is now set up so that subsequent FACs are automatically archived correctly. SandyGeorgia (Talk) 17:15, 16 November 2007 (UTC)

I agree that postponing the archiving to subsequent peer reviews is a recipe for trouble: subsequent editors will not (in general) fix the links, even if the instructions ask them to (which they don't at the moment). (The only way to make nominators do something is to make the nomination fail until they do it!) So we either need archiving by bot, or a permanent page ab initio. On this I am not completely clear how Gimmetrow's template code would work: the template can certainly look for the next /reviewN page which is free, but once this page is edited, how to stop it looking for /reviewN+1? I am asking this because I want to know whether the code is compatible with the autogeneration of the PR page: the bot needs to know (through categories) which of the many /reviewN pages is the one currently under peer review. Geometry guy 21:37, 16 November 2007 (UTC)

"the ways I've thought to implement this would appear to require subst'ed templates"... to avoid looking for N+1. Somehow the active page would have to be a parameter in the template. Are you using categories from the talk page, or from the peer review page? If on the peer review page, they would be removed when the bot updates ArticleHistory or whatever when the PR is over. The bot could even put a note at the top of the finished PR saying that it's finished. Gimmetrow 21:45, 16 November 2007 (UTC)
Gguy, here's how GimmeBot currently does FACs, so you can see an example of changes Gimmetrow is discussing:
When a new nominator submits, the unarchived FAC page is ready and comes up, pre-loaded. When a FAC closes, the bot searches for the next open N in Featured article candidates/article name/archiveN and archives the current FAC there, clearing the redirect at the main FAC page, leaving it ready for the next submission. The nominator theoretically can't mess up (although amazingly, they sometimes find a way). If you look through those, you see how the bot leaves the pre-loaded FAC page ready for the next submission, by archiving to the next N when the previous FAC closes. You can find the steps in the process here. By archiving when the FAC closes, you leave less margin for error for the nominator. On the other hand, another problem is that not all editors wait for the bot, so they sometimes mess it up on the other end. SandyGeorgia (Talk) 21:53, 16 November 2007 (UTC)
Here is a typical sequence. The bot archived a peer review, then set it up for the next one. But it also updated links to this peer review, one in the PR archives,[3] and one in a FLC page.[4] Even if the PR page is not preemptively archived, once the page is moved to /archive1, at least these two links should be updated. Then the October archive points to the correct page, and future readers of the FLC page are pointed to the corresponding PR rather than a later one. (Two project pages also link to the review; those links were not updated.) If the template can pick an appropriate page when it's first added as suggested above, a bot wouldn't need to do so much. Gimmetrow 22:27, 16 November 2007 (UTC)

This is probably a stupid question, but I'll ask it anyway, otherwise I'll remain in ignorance. Would it be possible to combine that archiving scheme with transcluding the review into the article talk page? ---- Malleus Fatuarum (talk) 22:05, 16 November 2007 (UTC)

Probably, unless you mean something quite different than I imagine. You could even have the PR template do the transclusion and provide an edit link. Gimmetrow 22:27, 16 November 2007 (UTC)
Gimmetrow: I'm using categories from the talk page to autogenerate, so the review page number could probably be passed to a substituted template, which then puts the talk page in a category depending on the review page number. The problem is that we would have to list first PRs together, followed by 2nd PRs and so on. The bot I am using is a general purpose category listing bot, so if you can think of a cleverer way to use it let me know.
Sandy: thanks for the links. I agree with you. If we don't adopt Gimmetrow's idea, I think PR should follow the FAC/FAR practice that you describe, i.e., closed discussions are automatically moved to an archive page, clearing the main subpage for the next review request.
Malleus: Most of the schemes discussed are compatible with transcluding the peer review onto the talk page. The advantage of fixing the link permanently is that such a talk-page transclusion never needs to be changed. Geometry guy 22:23, 16 November 2007 (UTC)

I don't speak bot very well, and I'm losing track of where we are on this, so if anything changes, I hope someone will let me know. For now, when I review all FACs to prep for GimmeBot (and the errors are becoming fewer and fewer, just the daily GA articlehistory errors), if I find an oldpeerreview that wasn't moved to archive, I moved it to the next archiveN. SandyGeorgia (Talk) 22:10, 20 November 2007 (UTC)

When the bot handles closed FACs, it archives "oldpeerreview" where the subpage name matches the current article. You don't need to do it manually. The bot skips "peerreview" in case it's current. Gimmetrow 22:21, 20 November 2007 (UTC)
But that's what I mean (I think); often current PRs aren't closed when a FAC is started, and they should be. So when I moved then to oldpeerreview, I go ahead and archive. Should I only change the template to oldpeerreview and not archive? SandyGeorgia (Talk) 22:29, 20 November 2007 (UTC)
You can do the page moves if you want, but you don't need to if the template is changed to oldpeerreview. The bot will get it when it handles the FAC, as here. Gimmetrow 22:49, 20 November 2007 (UTC)
OK, since articles shouldn't be at both PR and FAC, when reviewing FACs, if I find an active PR, if I just remove it from PR and change the talk page template to oldpeerreview, that's enough? SandyGeorgia (Talk) 23:13, 20 November 2007 (UTC)
Currently, page moves are not done routinely. The *manual* process is: remove the transclusion from the PR page, add the transclusion to the current archive page (Wikipedia:Peer review/May 2024), and change the talk page template to oldpeerreview. If this isn't the case then Template:PR-instructions needs changing. DrKiernan (talk) 09:16, 21 November 2007 (UTC)
It sounds like you're correct, I'm doing more than I need to do, and GimmeBot can do the rest. SandyGeorgia (Talk) 15:13, 21 November 2007 (UTC)

It would be much better if the archiving process at PR also involved a page move, and the instructions were updated to reflect this. However, since this imposes more work on archivists, I suggest waiting for Gimmetrow to adapt the FAC bot to PR, so that the extra work (and more) is automated. Geometry guy 21:03, 21 November 2007 (UTC)

A side note on LoCE

Just spotted this at LoCE. Interesting parallels to some of our discussions. EyeSerene has invited some LoCE participants over to this workshop; I don't know if LoCE really falls within our remit, but I thought I'd point it out. Mike Christie (talk) 01:29, 6 December 2007 (UTC)

The copyediting thing is very interesting, and I think very much ought to be in our remit. Almost by definition, the only stable points in an article's life are when it's nominated for either GA or FA. So surely that's when the limited resources of the LoCE could be best deployed. Why would anyone want to invest the effort to copyedit an article that may change out of all recogniton in a couple of weeks and need to be copyedited all over again? --Malleus Fatuorum (talk) 01:52, 6 December 2007 (UTC)
Been there, done that, got no T-shirts. Have pretty much decided not to copy-edit any articles except GAC and FAC, for exactly those reasons. Unimaginative Username (talk) 04:03, 6 December 2007 (UTC)
Having just read this, I'm encouraged to think that I'm not alone in my belief. --Malleus Fatuorum (talk) 02:16, 6 December 2007 (UTC)
As all the sci-fi movies about extra-terrestial life say, "You are not alone". Unimaginative Username (talk) 04:03, 6 December 2007 (UTC)
I'll second (sorry, third) this.....FA and GA are great as reference points...cheers, Casliber (talk · contribs) 06:02, 6 December 2007 (UTC)
I'm in 100% agreement with all the above; it was partly the discussions Mike linked to above that prompted me to alert the LoCE, as the same things we're discussing seem to be cropping up elsewhere too. It makes sense to collaborate with like-minded editors, and poor article prose is almost invariably mentioned in reviews. To take Malleus's point, all the projects have limited resources, and unless we can find ways of deploying them more intelligently we're all going to drown ;) EyeSereneTALK 11:53, 6 December 2007 (UTC)
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia_talk:Content_review/workshop/Archive_1&oldid=1090630325"