[Tccc] ComSoc technical cosponsorship - r...

Xiaoming Fu fuatcs.uni-goettingen.de
Mon Jun 3 03:27:44 EDT 2013



 I agree with you, Mark. Regional/national conferences make a particular 
complementary role in filling the cost/distance gap for many people.  

For international conferences at large, an exact rule based ranking does 
only partial job in ranking them, due to the faking possibilities that 
colleagues have already pointed out. One idea could be to introduce kind 
of online reputation system platform, where past attendees can comment 
on and rank the conference from multiple aspects (single/multi-track, 
presentation quality, attendee interaction, even food/hotel quality 
etc). Anyone who has not been involved or attended the event can look at 
these comments, just like what Amazon or Ebay platform does. Community 
feedback-based ranking might make more sense, one can discern from whom 
have attended and what they have comments about. One has to care also 
about the trustworthiness of such online system, avoiding coordinated or 
uncoordinated "shills".
Xiaoming

On 6/3/2013 5:56 AM, Mark Gregory wrote:
> Hi,
>
> great discussion.
>
> If I may highlight there are two types of conferences that we need to 
> assist - though for some people the second type of conference may be a 
> concern. The first are the international conferences that are bid for 
> and move to a different country each year. Plain enough these can be 
> identified and ranked. The second are the regional or national 
> conferences that move between cities in one country. They provide a 
> low cost way for students to publish papers and as long as the TPC is 
> strong the papers are of reasonable strength.
>
> I have been involved with ATNAC for 15 years and it is vital for 
> Australia / New Zealand because of the cost of going to international 
> conferences in Europe and US.
>
> ATNAC is just an example of the second type of conference - every 
> region has their local conferences and I hope the explosion of 
> "international" conferences don't cause the local conferences to be 
> lost in the ether.
>
> Regards,
> Mark Gregory
>
> BEng(Elec)(Hons), MEng, PhD, FIEAust, SMIEEE
> Senior Lecturer
> School of Electrical and Computer Engineering
> RMIT University
> CRICOS: 00122A
>
> Phone:  +61 3 99253243 Fax: +61 3 9925 2007
> GPO Box 2476, Melbourne, Victoria, 3001, AUSTRALIA.
> mark.greg... at rmit.edu.au <mailto:mark.greg... at rmit.edu.au>
>
> CONFIDENTIALITY NOTICE:  This email message, including any 
> attachments, is for the sole use of the intended recipient(s) and may 
> contain confidential and privileged information.  Any unauthorised 
> review, use, disclosure or distribution is prohibited.  If you are not 
> the intended recipient, please contact the sender by reply email and 
> destroy all copies of the original message.
>
>
>
> On 3 June 2013 12:44, <ddee... at mail.gov.mu 
> <mailto:ddee... at mail.gov.mu>> wrote:
>
>     I support Anthony.it is a waste of time and resources.
>     DD
>     Sent from my BlackBerry wireless device
>     ------------------------------------------------------------------------
>     *From: * Xiaoming Fu <f... at cs.uni-goettingen.de
>     <mailto:f... at cs.uni-goettingen.de>>
>     *Date: *Sun, 2 Jun 2013 23:13:11 +0200
>     *To: *<i... at comsoc.org <mailto:i... at comsoc.org>>
>     *Subject: *Re: [InternetTC] [Tccc] ComSoc technical cosponsorship
>     - rating the review process
>
>     In the recent ITC practice, we ask 2(+) TPC members of each
>     conference
>     endorsed by ITC to report the statistics (#submitted, #accept,
>     #participants, etc). In the upcoming ITC meeting at ICC'13 we will
>     give an
>     overview of these conferences and hope to get some feedback on
>     possible
>     improvements too.
>     Xiaoming
>
>     On 5/31/2013 12:00 AM, Joe Touch wrote:
>     > Hi, Anthony,
>     >
>     > The point of this is to support a bureaucratic process - TCS,
>     and TC
>     > endorsement. Right now, endorsement requires naming two TC
>     members who
>     > will oversee a meeting. There is no suggestion to what kinds of
>     > information they will report back to the TC.
>     >
>     > This isn't intended as a fixed form. TC members can report anything
>     > they want. TCs can ask anything they want. This is just intended
>     as a
>     > place to start.
>     >
>     > I appreciate that others on this list have various views of whether
>     > this can or should be a TC or ComSoc doc. I'll leave that to the TC
>     > chairs or the ComSoc to decide.
>     >
>     > Regardless of whether they decide to move forward with this or
>     "scrap"
>     > it, I will be posting it on a site I maintain for conference best
>     > practices anyway, FWIW.
>     >
>     > Joe
>     >
>     > On 5/30/2013 2:09 PM, Anthony Ephremides wrote:
>     >> I think this whole idea is another example of bureaucratic
>     thinking
>     >> within the Society. Any such ranking will be fraught with
>     >> inaccuracies and will convey erroneous messages. We all know a
>     good
>     >> conference when we see one. Rejection rates as a metric of
>     quality??
>     >> Holding a TPC meeting as a measure of quality? Who will measure
>     the
>     >> quality of the reviews and of the authors?
>     >> My suggestion is to scrap the project.
>     >>
>     >> AE
>     >>
>     >>
>     >>
>     >>
>     >>
>     >> Anthony Ephremides
>     >> Distinguished University Professor and
>     >> Cynthia Kim Eminent Professor of
>     >> Information Technology
>     >> ECE dept and ISR
>     >> University of Maryland
>     >> College Park, MD 20742
>     >> 301-405-3641 <tel:301-405-3641>
>     >> etony(at)umd(dot) edu
>     >>
>     >>
>     >> -----Original Message-----
>     >> From: Henning Schulzrinne [mailto:h... at cs.columbia.edu
>     <mailto:h... at cs.columbia.edu>]
>     >> Sent: Thursday, May 30, 2013 2:33 PM
>     >> To: Ken Calvert
>     >> Cc: Tccc at lists.cs.columbia.edu
>     <mailto:Tccc at lists.cs.columbia.edu>; tc... at ri.uni-tuebingen.de
>     <mailto:tc... at ri.uni-tuebingen.de>;
>     >> i... at comsoc.org <mailto:i... at comsoc.org>; Joe Touch
>     >> Subject: Re: [Tccc] ComSoc technical cosponsorship - rating the
>     >> review process
>     >>
>     >> Also, ranking could be seen as meaning that a TPC member ranks
>     papers
>     >> within their review portfolio (paper #7 is best, #8 second best").
>     >> (Infocom tried this, I believe.) I don't think that works all that
>     >> well, but "ranking" may well refer to the usual "definite
>     accept" to
>     >> "definite reject" scale or "in top 10% of papers". Given the
>     tendency
>     >> of the first ranking to concentrate around the non-committal
>     middle,
>     >> the latter seems more helpful, but I'm not sure that's a "best"
>     >> practice.
>     >>
>     >> On May 30, 2013, at 11:40 AM, Ken Calvert
>     <calv... at netlab.uky.edu <mailto:calv... at netlab.uky.edu>>
>     >> wrote:
>     >>
>     >>> Hi Joe -
>     >>>
>     >>> Good idea, thanks for doing this. I think your proposal is pretty
>     >>> much on target. Just a couple of thoughts on #6:
>     >>>
>     >>>> 6. paper review process E/A/D
>     >>>>
>     >>>> E = considers average rank AND outlier info, discussion points
>     >>>> also based on natural 'gap' in evaluation
>     >>>> A = considers average rank based on natural gap in evaluation
>     >>>> D = considers rank only
>     >>>
>     >>> (i) I interpret these criteria as referring to the accept/reject
>     >>> decision process, rather than the "paper review process". Perhaps
>     >>> the title should be "acceptance decision process" or something
>     like
>     >>> that?
>     >>>
>     >>> (ii) What about considering the transparency of the decision
>     process?
>     >>> I.e., whether all (or almost all) decisions are made with in full
>     >>> view of the TPC and with the TPC's approval or at least the
>     >>> opportunity to object.
>     >>>
>     >>> (iii) Can you please clarify what you mean by "natural gap in
>     >>> evaluation"? I would probably interpret this to mean that the
>     >>> accept/reject line is drawn, as far as possible, so that there
>     is a
>     >>> clear gap between the (average ratings of) the accepted papers
>     and
>     >>> the rejected papers. But I don't think that's realistic -
>     >>> especially in large/general conferences, where there are
>     papers from
>     >>> many areas, there will be not be a bright line in the
>     >>> ratings/rankings between rejected and accepted papers. This also
>     >>> seems to conflict with "considers rank only" being Deficient. So
>     >>> maybe I've just not understood what this means.
>     >>>
>     >>> Cheers,
>     >>>
>     >>> KC
>     >>>
>     >>> On 29 May 2013, at 14:05 PM, Joe Touch <to... at isi.edu
>     <mailto:to... at isi.edu>> wrote:
>     >>>
>     >>>> Hi, all,
>     >>>>
>     >>>> As part of the ComSoc technical cosponsorship (TCS) process,
>     TCs are
>     >>>> supposed to nominate at least two members of the TPC who will
>     monitor
>     >>>> the review process.
>     >>>>
>     >>>> However, there doesn't appear to be any guidelines for providing
>     >>>> feedback on that process.
>     >>>>
>     >>>> I've drafted the following, which I hope will open a
>     discussion on
>     >>>> this issue. If it evolves into something useful, perhaps it
>     can be
>     >>>> posted on the TC websites for use by those appointed to monitor
>     >>>> TC-endorsed TCS'd meetings.
>     >>>>
>     >>>> NB: I've cross-posted this to TCCC, ITC, and TCHSN, which are
>     where I
>     >>>> participate primarily; if any other TC has suggestions,
>     please take
>     >>>> the discussion to the TCCC list if possible.
>     >>>>
>     >>>> Thanks,
>     >>>>
>     >>>> Joe
>     >>>>
>     >>>> -----------------------------------------------------------------
>     >>>>
>     >>>> Rating system:
>     >>>> EXCELLENT best-practice to be aspired to
>     >>>> AVERAGE acceptable practice
>     >>>> DEFICIENT cause for concern for ComSoc involvement
>     >>>>
>     >>>> 1. TPC participation invitation E/A/D
>     >>>>
>     >>>> E = before first Call for Papers (CFP) issued
>     >>>> A = before CFP submissions due
>     >>>> D = after CFP submissions due
>     >>>>
>     >>>> 2. involvement in CFP promotion E/A/D
>     >>>>
>     >>>> E = invited to forward CFP and submit
>     >>>> A = invited to submit
>     >>>> D = neither
>     >>>>
>     >>>> 3. paper assignment for review E/A/D
>     >>>>
>     >>>> E = invited to select papers based on expertise and
>     >>>> abstracts/titles
>     >>>> A = invited to select based on topic area
>     >>>> D = not invited to select
>     >>>>
>     >>>> NB: "everyone reviews all" = E
>     >>>>
>     >>>> 4. paper review format E/A/D
>     >>>>
>     >>>> E = includes rank, feedback for author, and private
>     >>>> feedback for TPC discussion
>     >>>> A = includes rank and author feedback
>     >>>> D = includes only rank
>     >>>>
>     >>>> 5. TPC meeting E/A/D
>     >>>>
>     >>>> E = in-person meeting with support for remote
>     >>>> A = in-person with no remote support or only telecon or e-mail
>     >>>> D = no meeting
>     >>>>
>     >>>> 6. paper review process E/A/D
>     >>>>
>     >>>> E = considers average rank AND outlier info, discussion points
>     >>>> also based on natural 'gap' in evaluation
>     >>>> A = considers average rank based on natural gap in evaluation
>     >>>> D = considers rank only
>     >>>>
>     >>>> 7. paper reviews returned E/A/D
>     >>>>
>     >>>> E = >=3 substantive reviews returned with rank and
>     >>>> comments for the authors
>     >>>> A = >=3 substantive reviews returned with rank and
>     >>>> at least a rationale for rejects
>     >>>> D = <3 reviews for some papers, reviews not returned at all,
>     >>>> or only rank provided
>     >>>>
>     >>>> 7. paper accept rate E/A/D
>     >>>>
>     >>>> E = <=50%, based on natural gap in paper evaluation
>     >>>> A = <=50%, not based on 'gap'
>     >>>> D = >50%
>     >>>>
>     >>>> ------------------------------------------------------
>     >>>> _______________________________________________
>     >>>> IEEE Communications Society Tech. Committee on Computer
>     >>>> Communications
>     >>>> (TCCC) - for discussions on computer networking and
>     communication.
>     >>>> Tccc at lists.cs.columbia.edu <mailto:Tccc at lists.cs.columbia.edu>
>     >>>> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>     >>>
>     >>> Ken Calvert
>     >>> Professor and Chair, Computer Science Department Acting
>     Director, Vis
>     >>> Center University of Kentucky
>     >>>
>     >>>
>     >>>
>     >>>
>     >>>
>     >>> _______________________________________________
>     >>> IEEE Communications Society Tech. Committee on Computer
>     Communications
>     >>> (TCCC) - for discussions on computer networking and communication.
>     >>> Tccc at lists.cs.columbia.edu <mailto:Tccc at lists.cs.columbia.edu>
>     >>> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>     >>>
>     >>
>     >>
>     >> _______________________________________________
>     >> IEEE Communications Society Tech. Committee on Computer
>     Communications
>     >> (TCCC) - for discussions on computer networking and communication.
>     >> Tccc at lists.cs.columbia.edu <mailto:Tccc at lists.cs.columbia.edu>
>     >> https://lists.cs.columbia.edu/cucslists/listinfo/tccc
>     >>
>

_______________________________________________
IEEE Communications Society Tech. Committee on Computer Communications
(TCCC) - for discussions on computer networking and communication.
Tccc at lists.cs.columbia.edu
https://lists.cs.columbia.edu/cucslists/listinfo/tccc
 




More information about the Tccc mailing list