What we need are some definite proposals about precisely those categories of evidence.
It seems to me that there are two appropriate criteria:
(1) Market-based -- recognition through market-based publication, etc.
(2) Credential-based -- recognition through degrees, scholarly publication, etc.
I'll use myself as a case in point. I'm not terribly well credentialed, compared to some of the Ph.Ds here. I have an MA in Creative Writing, but I'm Ab.D (absent dissertation) in English lit. I did pass Ph.D written and oral exams in Medieval, Renaissance, Restoration/18th Century, Southern American, and 20th Century literature, so I have a pretty strong background in literature topics. But, I'd argue, my Market-based credentials are stronger than those of many tenured professors. I've published three commercial books, all but the newest of which have gone into multiple printings (one is in its ninth printing and still in print 15 years after it appeared). For years I worked as a professional book editor, and collaborated with high-powered commercial writers on many successful books, including one that was a NYT best-seller. I've got strong credentials as a writing teacher, and have won awards for poetry and fiction. I'm a good prose stylist. So, while I might not have the scholarly mojo to claim familiarity with the cutting-edgiest scholarship, I do have superior qualifications as an editor of experts' writing, a sense of what the audience requires, and so forth. The overall package, I'd say, is a good one for an editor.
Compare that, now, to--say--someone who has the Ph.D, and perhaps the tenure-track appointment, but is not a native English speaker, whose expertise is in a fairly obscure corner of the academic landscape, and who writes rather stilted academic prose. Which one should be trusted to be responsible for an interesting, coherent, and accessible encyclopedia entry aimed at a general reader? I might know less about a given subject, but in many cases my market experience might make me a better editor for the final entry.
I think the challenge is going to be in figuring out a way to balance the two criteria.
--Robert Rubin