Thoughts on arXiv's latest survey

arXiv is one of the main resources for disseminating physics and mathematics papers to researchers and the general public. Papers on arXiv are freely available without registration. They often have minor differences to the official published version but only at the level of typesetting, not actual content. They recently conducted a user survey, of which the results are now available.

I just have a couple of comments on the results :
Several respondents said they were unaware of precisely what quality-control measures were already in place, and felt that the process is too opaque. Others acknowledged the difficult balance between rejecting papers that are clearly unworthy—“crackpot”—and rejecting papers for other, perhaps less obvious, and anonymized reasons. However, even in the face of such criticisms there was a strong thread of satisfaction with arXiv’s current quality-control process and users cautioned against going too far in the other direction.

Personally I wrote very strongly that I don't want to see arXiv acting as another referee. arXiv should be about communication and dissemination, not quality control. That should be the job of the referees, but papers should be carefully labelled as to whether they're accepted to a journal (or conference proceedings or whatever), submitted but under review, or only available on arXiv. arXiv's quality control should be strictly limited to controlling obvious spam (e.g. advertising), not deciding what's scientifically valuable or not.

The level of crackpottery on arXiv is so low that it's still entertaining, and it's anyway useful to give such people a voice so that they can't be accused of being silenced. It's even quite useful to get a feel for what people object to about standard theories in order to be prepared for it elsewhere. Filters for journal status (which arXiv needs some way of verifying) would cut this down completely for anyone feeling less than amused.

The idea of adding an annotation feature to allow readers to comment on papers was almost evenly split, with 34.89% of users ranking it as very important/important and 34.08% as not important/should not be doing this. In the open text responses, the trend opposed the idea and some of the responses reflected strongly negative feelings. Those in favour or open to the idea of a commenting system often added a caveat and in general there was a sense of caution even for those responding positively.

I would really like a forum with each paper automatically (with the author's permission) starting a thread, but only if it was very carefully moderated. Topics should be limited to that paper and related research, otherwise you'll get hordes of people who just want to start an argument. I think it could be a great way for the public to see scientists debating each other, and potentially for direct outreach too. The problem is that arXiv is huge, so this would probably be a full-time job for several people.
https://confluence.cornell.edu/display/culpublic/arXiv+User+Survey+Report