Rob, quite seriously, these are important questions and need to be thought about and talked about. Even so, I know just enough about qualitative research methodology to know that there's almost no way to develop a sound evaluative instrument that provides a robust sample and also protects respondent anonymity in a typical dojo setting.
I think that best practices can be identified and shared usefully, but I have real doubts about their universality or the extent to which they can be codified.
You express my thoughts so much better than I can!!
Well, Dan speaks of a vetting system. What is said about someone that gets them no access? "He's a jerk", "He's all talk and isn't serious about training", "He just wants to be your fanboy", "This person is abusive with power.", etc.
Seems like we have some questions we use today - they just aren't codified and generalized. Even if it can't be done perfectly, I'd still say we are better off trying to come as close as we can.
I understand that some jerks are helpful. But could they have been helpful and not have been jerks?
(darn - I have more to say about this but I'm getting interrupted...)