[Tccc] Need co-workers (Re: Requesting open feedback to my work (Re: Promoting open on-line research))

Martin Gilje Jaatun atc08
Thu Nov 3 17:21:41 EDT 2011


On 11/03/2011 06:24 PM, Pars Mutaf wrote:
> Hi all,
>
> If you like the following idea, have similar ones, suggest modifications,
> helpful strategies to get it done please contact me (pars.mutaf at gmail.com).
It seems to me that a good place to start would be something like this:
http://myreview.sourceforge.net/

-Martin
> A system providing the following:
>
> 1. I can browse others' scientific work (e.g. arxiv.org)
> 2. I can ask questions, provide comments, get answers etc.
> 3. My input is archived.
> 4. I get comments to my work. If I don't there is a problem with my work
> and I update it or see similar work.
>
> Using this system, if I provide good feedback, I can form a network for
> myself without necessarily attending conferences.
>
> ===
> Pars Mutaf
>
>
>
> On Thu, Nov 3, 2011 at 6:11 PM, Pars Mutaf<pars.mutaf at gmail.com>  wrote:
>
>> Hi Joe thanks. I think I cannot argue with your experience of course which
>> I don' have.
>>
>> But why the following system is not useful to me?
>>
>> 1. I can browse others' work (e.g. arxiv)
>> 2. I can ask questions, provide comments, get answers etc.
>> 3. My input is archived.
>> 4. I get comments to my work. If I don't there is a problem with my work
>> and I update it or see similar work.
>>
>> Using this system, if I provide good feedback, I can form a network for
>> myself without necessarily attending conferences.
>>
>> Pars
>>
>>
>>
>> On Thu, Nov 3, 2011 at 5:44 PM, Joe Touch<touch at isi.edu>  wrote:
>>
>>>
>>> On 11/3/2011 7:00 AM, Pars Mutaf wrote:
>>> ...
>>>
>>>   The reviews should be publicly available to everyone.
>>> There have been attempts to explore this and other models, e.g., in no
>>> particular order:
>>>
>>>         A- author rebuttal of reviews
>>>         B- blind reviews
>>>         C- double-blind process
>>>                 where the paper authors are hidden during review
>>>         D- public reviews
>>>                 where reviews are published with the paper
>>>         E- open reviews
>>>                 where the author sees the reviewer's names
>>>         F- adding a venue for papers on the 'borderline' of the
>>>            main conference
>>>
>>> Speaking as someone who has participated as a PC member in these in
>>> various places (as an individual, not as TCCC Chair):
>>>
>>> A was tried at Infocom (and elsewhere). The goal was to avoid a paper
>>> being discarded because of an incorrect review. The result was a
>>> substantial increase in review time (actually, it ended up resulting in
>>> less time for reviewers to complete their reviews due to a fixed yearly
>>> cycle), but no substantial change in paper handling. Most of the rebuttals
>>> did not point out review errors, but rather disagreed with review opinion.
>>>
>>> B is currently typical.
>>>
>>> C is used at Sigcomm and more recently at ICNP. It is intended to avoid
>>> favoritism, but IMO it also tends to work against systems work that has
>>> been vetted in workshops and symposia in parts.
>>>
>>> D has been tried for some CCR papers, where a single review or summary of
>>> the reviews is presented.
>>>
>>> E was tried at Global Internet a number of years ago, and nearly killed
>>> the meeting. Submissions went down over 50%. The result was much more
>>> pleasantly-written reviews, but the reviews were (IMO) less useful.
>>>
>>> F was introduced at Infocom several years ago. IMO, it simply introduced
>>> a second borderline, and made it very difficult to distinguish between full
>>> accepts and "consolation prize" accepts.
>>>
>>> All of the above were introduced to address a perceived or real concern.
>>> None of them was tested in a true experiment (e.g., with a control group
>>> during the same year). Most of them (IMO) were introduced because chairs
>>> believe that mechanism can address review process problems. IMO, there is
>>> only one good solution for all such problems:
>>>
>>>         PC chairs MUST review the reviews. EVERY review. EVERY year.
>>>                 Reviews whose ranks are not substantiated by
>>>                 meaningful comment must be both discarded and
>>>                 replaced.
>>>
>>> Overall, IMO, it is useful to understand that:
>>>
>>>         - reviewing is an imperfect process
>>>
>>>         - a paper's quality is determined by what the reader
>>>           receives (goodput), not what is sent (offered load) ;-)
>>>
>>>         - papers are rejected because of the lack of positive comments,
>>>           not for any single negative comment
>>>                 (so arguing each negative comment in a review
>>>                 won't fix a paper - many reviewers simply provide
>>>                 sufficient negatives to justify a decision, but
>>>                 could provide other negatives if asked)
>>>
>>>         - at large conferences, papers are rejected after substantial
>>>           decision
>>>                 e.g., at Infocom, a paper is either a unanimous reject
>>>                 by three reviewers, OR is then considered by at least
>>>                 an additional 8-10 people during the PC meeting
>>>
>>> I see none of these changing in an open process.
>>>
>>> Joe
>>>
>>>
>>>
> _______________________________________________
> IEEE Communications Society Tech. Committee on Computer Communications
> (TCCC) - for discussions on computer networking and communication.
> Tccc at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>




More information about the TCCC mailing list