have Larry et. al. stated that Citizendium could have avoided Essjay problem?

[bennett]

I wrote on article previously for Slashdot comparing Wikipedia and Citizendium, arguing for the Citizendium side on the grounds that articles attributed to specific, credentialed people were more valuable to people who wanted to cite them, for example.  Obviously the Essjay controversy has brought this issue into the public eye.  (Although technically the issues are different -- the Essjay flap is about how closely someone's credentials should be scrutinized, whereas the main contrast between Wikipedia and Citizendium is about how much value an article has when it is not attributable to *any* individual, whether credentialed or not!)

I may be writing a follow-up and had some questions:

1) I remember that at some point, the FAQ at http://citizendium.org/faq.html said that users' self-identification worked on "the honor system".  This now seems to be gone from the FAQ; the FAQ emphasizes what role credentials will play but doesn't say whether those will be verified.  Do you, or do you plan to, verify the credentials of contributors?

While this is sometimes hard, I can think of one form of "verification" which would be very easy; if someone claims to be a university professor, just send a confirmation link to their e-mail address at their .edu domain.  (This assumes that either (a) email addresses at that domain are structured so that you can tell the difference between student and faculty e-mails, or (b) you can find a web page in the faculty section of the university's website, which identifies that address as the e-mail address of a faculty member.)

For the record, I think that verifying credentials where possible would be a good thing, for the same reason that attributing articles to authors is a good thing in the first place: Because it increases the value of the articles to people who (a) want to cite them, or (b) want to have a high degree of certainty that the article is accurate.

2) IF Citizendium verifies credentials, or is moving in that direction, then has Larry or anybody else ever publicly pointed out, in the Essjay controversy, that the Citizendium model could have avoided that problem through verification?

I read Larry's blog posts but couldn't find any statement to that effect.  The post at
http://blog.citizendium.org/2007/03/01/wikipedia-firmly-supports-your-right-to-identity-fraud/
says that Citizendium is trying to avoid similar problems through measures such as term limits.  But it doesn't mention avoiding problems by verifying credentials.

[Larry Sanger]

Bennett, you might hold off on your follow-up for a bit.  We're going to do a public launch I hope next week, and if not then, the week after.  It all depends on when some new servers are delivered by a vendor to Steadfast Networks.

1) I remember that at some point, the FAQ at http://citizendium.org/faq.html said that users' self-identification worked on "the honor system".  This now seems to be gone from the FAQ; the FAQ emphasizes what role credentials will play but doesn't say whether those will be verified.  Do you, or do you plan to, verify the credentials of contributors?

Yes.  Please see this blog post.

It was never the case that we would "simply trust" editors' claims.  We began with the utopian idea that editors would self-identify, but that everyone would check out the links that the purported editors provided on their user pages to establish their credentials.  Similar to Jimmy Wales' proposal, in fact.  But when we started inviting applications from people for editor positions in the pilot project, we discovered that a large minority of applicants were actually not qualified to be editors, and many were very grumpy about being excluded as editors.  So we thought there would be endless problems if we let people self-identify as editors--which is why we decided to hand-approve them.

As to authors, though, it's true that, until only about a month ago, we planned to have self-registration.  What we didn't realize, when we (temporarily) switched to self-registration in January and February, was just how motivated vandals would be.  They're quite motivated (and pathetic), as it turns out, which means that we have a choice: either constantly babysit idiot vandals, or else screen applicants by hand.  We decided for the latter.

What we're going to do next, when we have coded up a new semi-automated registration system (which will make it possible for constables to get new people on board with the push of a button), is detailed in the above-linked blog post.

While this is sometimes hard, I can think of one form of "verification" which would be very easy; if someone claims to be a university professor, just send a confirmation link to their e-mail address at their .edu domain.  (This assumes that either (a) email addresses at that domain are structured so that you can tell the difference between student and faculty e-mails, or (b) you can find a web page in the faculty section of the university's website, which identifies that address as the e-mail address of a faculty member.)

We'll do something vaguely similar to this.  At least to begin with, we won't be quite as careful as this suggests.  We will be doing about the same amount of work in checking credentials that, I imagine, any academic project online would do.

For the record, I think that verifying credentials where possible would be a good thing, for the same reason that attributing articles to authors is a good thing in the first place: Because it increases the value of the articles to people who (a) want to cite them, or (b) want to have a high degree of certainty that the article is accurate.

We agree, of course.

2) IF Citizendium verifies credentials, or is moving in that direction, then has Larry or anybody else ever publicly pointed out, in the Essjay controversy, that the Citizendium model could have avoided that problem through verification?

The Citizendium model would have avoided the problem for the simple reason that, for editorship, we do require a CV and Web link(s) that support claims made in the CV--and we look at the e-mail address, too.  If Essjay were to try to apply for editorship, he would have had to give us his real name (or what he purported to be his real name), construct an elaborate academic CV (real tenured professors have them ready to hand), and point to a credible source online that supports what that name says.  If you simply impersonate an actual, named professor, you are actually breaking a law.  If Essjay decided not to apply for editorship, then it would have been easier to commit his fraud (but still, he would have had to make a more elaborate lie for us, in offering what he claimed to be his real name).  Furthermore, according to the rules we will adopt when the semi-automated system is online, he would have had to jump through more hoops, quite easy to clear if you are who you claim to be, but harder if you're a liar.  Now, we know that there are loopholes in these procedures, but they make liars spin particularly elaborate lies.  Insofar as we trust that people will not go to that much trouble--actually to exploit these loopholes--we are still relying on an honor principle.

By the way, while I realize of course that there's a difference between vandalism and fraud, it's worth pointing out that both before and after our self-registration period, we had ZERO vandalism and very few behavior problems.  Establishing a community that is more mature and responsible will by itself tend to rein in various kinds of abuse, I think.

There's another way in which the Essjay situation would certainly never be repeated on CZ.  It's that, if he were to go through our registration process and then it were discovered that he were a fraud, he would be instantly banned, regardless of how wonderful everybody thought he was.  Moreover, we would investigate what (if any) legal recourse we might take against him.

Also, um, after his fraud were discovered, he wouldn't be offered a job.  And, he wouldn't be named to the highest judicial tribunal of the project by the editor-in-chief.  And the editor-in-chief wouldn't defend him to The New Yorker by saying he didn't really have a problem with such fraud.  So there's that.

[Nancy Sculerati]

I  think that it must be acknowledged that in verifying credentials there is always a range of certainty, that has to do with  the depth of investigation.

There is what appears to be a completely loose situation in which there is no requirement for even giving an identity, and no check (WP?) - to us - to "Top Secret FBI Unipol clearance "  .

My point? Even when there is snail mail document delivery and phone call  verifications, there are people who are outed only years after having made up their credentials. That happens throughout the world and we are not immune.   

Nancy Sculerati MD (Citizendium user name)

[Stephen Ewen]

While this is sometimes hard, I can think of one form of "verification" which would be very easy; if someone claims to be a university professor, just send a confirmation link to their e-mail address at their .edu domain. 

As a constable who processes applications for authors, I do something like this routinely.  So, if someone applies with their gmail account and saying they are a student at a university - you get the idea.

[Zachary Pruckowski]

While this is sometimes hard, I can think of one form of "verification" which would be very easy; if someone claims to be a university professor, just send a confirmation link to their e-mail address at their .edu domain. 

As a constable who processes applications for authors, I do something like this routinely.  So, if someone applies with their gmail account and saying they are a student at a university - you get the idea.

Strangely enough, I fit exactly that description

[bennett]

If changes have to be approved by an editor, then as long as you trust the editor, what damage do the vandals really do?  They submit garbage chances and the editor rejects them; surely the editor is annoyed by the spam, but do the vandals do more damage than that?

Or are you talking about damage done during the initial building-up process of a new article, when the changes submitted by a vandal actually change the content as viewed by everybody.

It seems like there are two problems here:

1) Damage done to Citizendium's credibility by the possibility that an author lied about their credentials.  But once an article gets to the stage where an editor signs off on the accuracy of the contents, isn't it true that at that point, only the editor's credentials matter as far as credibility goes, not the credentials of the authors?

It depends on how someone would view an article if they knew, "The article's contents were written by an unverified person with a Hotmail account, but a professor of physics (whose .edu e-mail address we verified) signed off on its accuracy."  This is not really analogous to anything in old-world media, so to most people it would involve a new thought process.  But to me at least, it certainly seems logical to consider the content reliable.

So, as long as editors are thoroughly verified, it seems the credentials of authors would be a non-issue.  Am I missing something?

2) Damage done by blatant vandalism.  I can think of two ways to mitigate this:

- Have changes queued up to be reviewed before being submitted to an article, but this time they would be reviewed not for accuracy, but just to block obvious spam vandalism.  Since this is unskilled work, it could be done by any volunteer, not just the credentialed editor.  So turnaround time would be very fast, and it would lessen the burden on the credentialed experts, who are a scarce resource.

- Verify uniqueness of user identities by sending a postcard to their physical address with some six-digit code on it.  This makes it easy for one person to verify themselves, but much harder for one person to pretend to be multiple people.  You'd still have vandals, but they could only get away with vandalism once.

It seems both of these could be automated, and both of these would avoid the security problems described by Max and Tommy in response to Larry's blog post.

[Nancy Sculerati]

We cannot work in an atmosphere where we are being vanadalized constantly. There is no time to do more than revert vandalism, and there is loss of morale because much of the vandalism is personal insults. It would be like you trying to write an article (for slashdot?  ) with somebody constantly punching in random text on your keyboard while you are composing. the rate of vandalism, in our case, was just too great to allow us to accomplish our work.

Further, we are engaged in developing a unique internet culture here that prizes collaboration and respect. The development of that culture was particularly hurt by the vandalism. My view, anyway.

[bennett]

I'm sure you're right.  However I can't tell if this post is in support of the system as it currently is, or in support of the system as Larry proposes to change it, or in support of the counter-measures I proposed.

If changes are put into a queue to be reviewed in near-real-time by constables, then the "reward" for vandals is greatly decreased (it doesn't get seen by anybody but the constable), so you'd expect much less of it.

Similarly, if instead you verified new users by sending a postcard to their given address with a six-digit number on it, then most potential vandals would only get one crack at the system before they got banned, so you'd see much less vandalism that way as well.

[Derek Harkness]

- Have changes queued up to be reviewed before being submitted to an article, but this time they would be reviewed not for accuracy, but just to block obvious spam vandalism.  Since this is unskilled work, it could be done by any volunteer, not just the credentialed editor.  So turnaround time would be very fast, and it would lessen the burden on the credentialed experts, who are a scarce resource.

In effect that process is already enacted. However, it only goes into effect once once version of an article has been Approved. Once approval has happened, new edits to the draft don't instantly show up on the default pubic version. So the Approval system is IMHO our main weapon against vandalism. The more articles we approve, the harder it is to vandalise and the less effective any such vandal could be.

[bennett]

- Have changes queued up to be reviewed before being submitted to an article, but this time they would be reviewed not for accuracy, but just to block obvious spam vandalism.  Since this is unskilled work, it could be done by any volunteer, not just the credentialed editor.  So turnaround time would be very fast, and it would lessen the burden on the credentialed experts, who are a scarce resource.

In effect that process is already enacted. However, it only goes into effect once once version of an article has been Approved. Once approval has happened, new edits to the draft don't instantly show up on the default pubic version. So the Approval system is IMHO our main weapon against vandalism. The more articles we approve, the harder it is to vandalise and the less effective any such vandal could be.

True.  What I meant was to queue up the changes even before the article had been Approved.  The people who review and approve the changes would not have to be experts -- all they have to do is block the obvious spam vandalism from going through.  Once vandals realize their spam is not going to get through the queue, the incentive to vandalize would be greatly reduced.

Since anybody can approve changes as non-spam, the modifications at this stage could still proceed in almost-real-time.  The question is whether almost-real-time editing would create the same kind of organic article growth as the completely-real-time editing that exists now (and which is vulnerable to vandalism).

[Jason "Electrawn" Potkanski]

an "Edit Queue" is in development.

-jtp

[bennett]

Actually here's an idea that would be less work for moderators: Only the first one (or two or three) edits by a given user, have to go through the edit queue.  After that, if none of their edits get flagged as vandalism, then their future edits don't have to be queued up.

The nice thing about this is that if someone wanted to commit vandalism, they'd first have to contribute one (or two or three) useful edits that made it through.  Then once changes from that user were no longer moderated, they could start vandalizing things if they wanted to, but as soon as it got noticed, the changes would get backed out and that user would get canned.  Thus the only way that vandals would even get a chance to commit vandalism, would be to contribute something useful first, and even after they got booted, their net contribution to the project would still be positive.

And, far less work for moderators, if only the first N edits from a given user have to be reviewed.

[Stephen Ewen]

Actually here's an idea that would be less work for moderators: Only the first one (or two or three) edits by a given user, have to go through the edit queue.  After that, if none of their edits get flagged as vandalism, then their future edits don't have to be queued up.

The nice thing about this is that if someone wanted to commit vandalism, they'd first have to contribute one (or two or three) useful edits that made it through.  Then once changes from that user were no longer moderated, they could start vandalizing things if they wanted to, but as soon as it got noticed, the changes would get backed out and that user would get canned.  Thus the only way that vandals would even get a chance to commit vandalism, would be to contribute something useful first, and even after they got booted, their net contribution to the project would still be positive.

And, far less work for moderators, if only the first N edits from a given user have to be reviewed.

But don't you think that can be as easily gamed as WP's "3 Revert Rule"? Everyone can find a two typos to fix and replace a word with a synonym.

But really, substantive changes should first be hashed out on talk.

[Derek Harkness]

But really, substantive changes should first be hashed out on talk.

I don't think this applies to CZ as much as to WP. There are three states for articles here, approved, not approved and draft. If an article is approved, then only the editors can change that version, so it won't get vandalised. If it's not approved, then substantive changes need to be made so people should feel free to make these changes. If it is the draft version that is being edited, then go ahead and edit that freely. Any change to the draft will not appear on the main public end so if you mess up it doesn't matter so much. I can even conceive of drafts forking as authors write on different routes before an editor decides which fork is working out better.

I think it would be a shame if CZ ended up with the revert wars of WP. I think that unless it's obviously an error or vandalism or deletion, don't revert. If someone wants to reword or add to an article, they should feel free to do so. Any thoughts about undoing someone else's work should be hashed out on the talk page.

[bennett]

Actually here's an idea that would be less work for moderators: Only the first one (or two or three) edits by a given user, have to go through the edit queue.  After that, if none of their edits get flagged as vandalism, then their future edits don't have to be queued up.

The nice thing about this is that if someone wanted to commit vandalism, they'd first have to contribute one (or two or three) useful edits that made it through.  Then once changes from that user were no longer moderated, they could start vandalizing things if they wanted to, but as soon as it got noticed, the changes would get backed out and that user would get canned.  Thus the only way that vandals would even get a chance to commit vandalism, would be to contribute something useful first, and even after they got booted, their net contribution to the project would still be positive.

And, far less work for moderators, if only the first N edits from a given user have to be reviewed.

But don't you think that can be as easily gamed as WP's "3 Revert Rule"? Everyone can find a two typos to fix and replace a word with a synonym.

But really, substantive changes should first be hashed out on talk.

Well it could be gamed in that sense, but then the question is, has the "vandal" really had a net negative effect?  If they fix three genuine typos and then start vandalizing, they've contributed the help of fixing those three typos, and then the only negative is the amount of effort that it takes for a constable to spot the vandalism, revert it, and cancel the vandal's account, which is presumably not much.  So you've forced the vandal to make a positive contribution greater than their negative contribution.

And sooner or later, presumably, all the obvious typos would be weeded out, so it would get harder for vandals to game the system by fixing obvious errors, and then they'd have to make even *bigger* positive contributions, before getting the brief chance to make a negative contribution!

(To make sure the positive contribution outweighs the possibility of a negative, the initial edits should have to be improvements, even small ones such as fixing typos.  Changes that are merely neutral, such as replacing one word with a synonym, could be rejected as not making any improvement.)