[Tccc] Requesting open feedback to my work (Re: Promoting open on-line research)
Joe Touch
touch
Thu Nov 3 13:10:00 EDT 2011
On 11/3/2011 9:48 AM, Michalis Faloutsos wrote:
> As the co-chair of that Global Internet offering, I beg to differ with
> Joe's opinion.
>
> First, it is entirely subjective to claim that the fact that reviews
> were public reduced the number of submissions.
The statistics were that the submissions were significantly down that
one year, and it took several years to recover to pre-experiment levels.
My claim is that the experiment impacted the number of reviews, as that
was the only significant change that year, and no other workshop I know
of experienced a similar change (either at Infocom or elsewhere).
There are many possible reasons why this could occur:
1- authors didn't want to participate
we can't know why for sure
2- reviewers didn't want to participate
(actually, we know this happened, since a number
of previous PC members declined to participate
that year, and checked that the experiment wasn't
happening the following year before agreeing to
come back)
3- the process of conducting an experiment affected other
aspects of the conference (promotion, etc.)
We can't know for sure what the reason was. What we do know is that it
did happen.
> As an author, I would love to see who is reviewing my paper. I know a
> lot of people who would.
The question is whether the impact to the community is worth the cost,
and what you would do with that information if you had it (would it make
your paper better? would it make your future papers better?)
> One could also attribute this to the existence of mini-papers at
> infocom: still that is not a fact either.
While that's possible, GI has had a different focus from the bulk of
Infocom papers, and the miniconference from 2007 didn't have papers that
would have been typical for GI. Further, this impacted only GI at
Infocom, not the other workshops. That's fairly strong evidence that
this was GI-specific.
> The trickier part was to get TPC members that were bold enough to accept
> the openness.
Or interested in participating in a poorly designed experiment (no
control group, in particular).
> There were no problems doing that though.
That depends on whether you consider the drop in the number of previous
TPC members a problem, though.
> For more details, some data and outcomes were published here:
> http://ccr.sigcomm.org/online/?q=node/223
Yes, thanks for that link.
Joe
More information about the TCCC
mailing list