[Tccc] Need co-workers (Re: Requesting open feedback to my work (Re: Promoting open on-line research))
Martin Gilje Jaatun
atc08
Thu Nov 3 17:21:41 EDT 2011
On 11/03/2011 06:24 PM, Pars Mutaf wrote:
> Hi all,
>
> If you like the following idea, have similar ones, suggest modifications,
> helpful strategies to get it done please contact me (pars.mutaf at gmail.com).
It seems to me that a good place to start would be something like this:
http://myreview.sourceforge.net/
-Martin
> A system providing the following:
>
> 1. I can browse others' scientific work (e.g. arxiv.org)
> 2. I can ask questions, provide comments, get answers etc.
> 3. My input is archived.
> 4. I get comments to my work. If I don't there is a problem with my work
> and I update it or see similar work.
>
> Using this system, if I provide good feedback, I can form a network for
> myself without necessarily attending conferences.
>
> ===
> Pars Mutaf
>
>
>
> On Thu, Nov 3, 2011 at 6:11 PM, Pars Mutaf<pars.mutaf at gmail.com> wrote:
>
>> Hi Joe thanks. I think I cannot argue with your experience of course which
>> I don' have.
>>
>> But why the following system is not useful to me?
>>
>> 1. I can browse others' work (e.g. arxiv)
>> 2. I can ask questions, provide comments, get answers etc.
>> 3. My input is archived.
>> 4. I get comments to my work. If I don't there is a problem with my work
>> and I update it or see similar work.
>>
>> Using this system, if I provide good feedback, I can form a network for
>> myself without necessarily attending conferences.
>>
>> Pars
>>
>>
>>
>> On Thu, Nov 3, 2011 at 5:44 PM, Joe Touch<touch at isi.edu> wrote:
>>
>>>
>>> On 11/3/2011 7:00 AM, Pars Mutaf wrote:
>>> ...
>>>
>>> The reviews should be publicly available to everyone.
>>> There have been attempts to explore this and other models, e.g., in no
>>> particular order:
>>>
>>> A- author rebuttal of reviews
>>> B- blind reviews
>>> C- double-blind process
>>> where the paper authors are hidden during review
>>> D- public reviews
>>> where reviews are published with the paper
>>> E- open reviews
>>> where the author sees the reviewer's names
>>> F- adding a venue for papers on the 'borderline' of the
>>> main conference
>>>
>>> Speaking as someone who has participated as a PC member in these in
>>> various places (as an individual, not as TCCC Chair):
>>>
>>> A was tried at Infocom (and elsewhere). The goal was to avoid a paper
>>> being discarded because of an incorrect review. The result was a
>>> substantial increase in review time (actually, it ended up resulting in
>>> less time for reviewers to complete their reviews due to a fixed yearly
>>> cycle), but no substantial change in paper handling. Most of the rebuttals
>>> did not point out review errors, but rather disagreed with review opinion.
>>>
>>> B is currently typical.
>>>
>>> C is used at Sigcomm and more recently at ICNP. It is intended to avoid
>>> favoritism, but IMO it also tends to work against systems work that has
>>> been vetted in workshops and symposia in parts.
>>>
>>> D has been tried for some CCR papers, where a single review or summary of
>>> the reviews is presented.
>>>
>>> E was tried at Global Internet a number of years ago, and nearly killed
>>> the meeting. Submissions went down over 50%. The result was much more
>>> pleasantly-written reviews, but the reviews were (IMO) less useful.
>>>
>>> F was introduced at Infocom several years ago. IMO, it simply introduced
>>> a second borderline, and made it very difficult to distinguish between full
>>> accepts and "consolation prize" accepts.
>>>
>>> All of the above were introduced to address a perceived or real concern.
>>> None of them was tested in a true experiment (e.g., with a control group
>>> during the same year). Most of them (IMO) were introduced because chairs
>>> believe that mechanism can address review process problems. IMO, there is
>>> only one good solution for all such problems:
>>>
>>> PC chairs MUST review the reviews. EVERY review. EVERY year.
>>> Reviews whose ranks are not substantiated by
>>> meaningful comment must be both discarded and
>>> replaced.
>>>
>>> Overall, IMO, it is useful to understand that:
>>>
>>> - reviewing is an imperfect process
>>>
>>> - a paper's quality is determined by what the reader
>>> receives (goodput), not what is sent (offered load) ;-)
>>>
>>> - papers are rejected because of the lack of positive comments,
>>> not for any single negative comment
>>> (so arguing each negative comment in a review
>>> won't fix a paper - many reviewers simply provide
>>> sufficient negatives to justify a decision, but
>>> could provide other negatives if asked)
>>>
>>> - at large conferences, papers are rejected after substantial
>>> decision
>>> e.g., at Infocom, a paper is either a unanimous reject
>>> by three reviewers, OR is then considered by at least
>>> an additional 8-10 people during the PC meeting
>>>
>>> I see none of these changing in an open process.
>>>
>>> Joe
>>>
>>>
>>>
> _______________________________________________
> IEEE Communications Society Tech. Committee on Computer Communications
> (TCCC) - for discussions on computer networking and communication.
> Tccc at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>
More information about the TCCC
mailing list