I recently participated in a Facebook discussion about the Scientific American article by Mary Carmichael: All Together Now: Scientists Take Peer Review Public The discussion took place behind the doors of a closed FB group and was posed next to the question: "So is this Open Science?"
Is public peer review open science? I found myself drawn to the question of who constitutes the public audience and whether or not the "public" (whoever they might be) passively observed or actively participated in the publication review. I was somewhat surprised that the gist of the subsequent comments focused on the role that anonymity plays in peer review; the question of who is this public was addressed only to express concern that an active audience would provide something other than the civil, "evidence based" review provided by anonymous peers moderated by activist editors.
I've come to realize that its probably a bit hard to think about the ramifications of open public peer review because one giant issue consumes everyones thinking. The core argument for anonymity is pretty simple: anonymity insulates the scientific culture from the friction (aka 'static') produced in open debate. The argument moves serially to assert that anonymity is necessary to protect junior scientists from retribution by senior authors, whether that retribution be by exclusion or through subsequently negative peer review behind the closed doors of job evaluations or well, behind the anonymity of peer review. This reasoning has certainly been repeated elsewhere, probably like the passing of an oral history, and most recently observed by me in the comments (e.g.) on Peter Coles post / poll about peer review.
Holy cow. Look, I have no idea if the stupid mistakes I've made as an open peer reviewer are viewed that much worse by the offended authors than the stupid mistakes I've made in my papers. What I do know is that the intellectual contributions I've made to papers as its anonymous reviewer have yielded exactly ZERO career benefit compared to the papers I've written.
Anyway if you've been able to listen to me past the whine then you've come to the point of my post, which is that I wanted to reproduce some of my FB comments here. I'm asserting that the issue is not about absolutist (or even pragmatic) approaches to the current culture (where apparently retribution is an assumed behavior) but about changing the culture...
This argument is about culture and cultural norms. Cultures rarely change overnight so we can't very well ask that we drop anonymous peer review in favor of something else for some of the very reasons that you mention -- the current culture does not support it.
Open science is about acting to change current cultural norms and create new ones. These people [in the SA article] are acting this way for this reason; they aren't asking you to do so. They are acting to change the current culture where a senior scientist can destroy [behind closed doors] another scientist over valid disagreements, where overloaded editors are incapable of scientifically brokering referee/author debates, where reviewers get zero attribution for hours or days of effort put into peer reviewing of a paper, and where arbitrary actions by gatekeeper journals sculpt science in an unaccountable way. There are other things that they are trying to change that we haven't even mentioned: a culture that does not value or recognize public engagement through education and public outreach, a culture where data are wasted due to negative or unclear results..
And create new ones: time and time again "communities" have shown themselves capable of crafting expected norms of behavior and this is exactly what happens in science blog comments and forums. The "new articles comments" get winnowed away by a community that does not accept them.
Open science isn't trying to recreate the journal system either although they each believe in improving science through peer action. The journal system is what it is albeit one that exists entirely on the backs of unpaid, uncredited reviewers. Open science is proposing to create new systems (on the backs of unpaid,uncredited actors) that may also improve scientific communication and collaboration. The fact that the "rest of us" can see into the process is another important but also distinct goal.
