I think more focused workshops would be a good idea, but not more big generic conferences. For those, I think getting everyone to go to the same one is a good idea.

I like the annual meeting feel of ICSLP/Eurospeech, where acceptance rates are in the 80s and multiple papers are allowed and everyone in the field shows up.

The border case is highly variant. Let’s say a paper gets a 5,5,4

and is accepted. We could use a bootstrap-style analysis to say that the chance of a reviewer giving it a 5 is 2/3 and the chance of a 4 is 1/3 (let’s not worry about smoothing). What’s the chance of the paper getting

accepted if 5,5,4 is the minimum score?

Winning outcomes:

5,5,5 (2/3)**3

5,5,4 (2/3)**2 (1/3)

5,4,5 (2/3)**2 (1/3)

4,5,5 (2/3)**2 (1/3)

————————-

Total Chance: 74%!!

Now what about a paper with a 4,4,5 that would’ve

been rejected. Just swap 2/3 for 1/3 above and

the result is that it actually had a 26% chance of being

accepted. This means that we’re at about 75% F measure

even given the very conservative model. If we smoothed or used subsampling, the variance estimate would be higher.

Also note that the mean is pretty meaningless to this kind

of analysis. A 5,5,1 paper has a low mean, but still has a

2/3**3=8/27 (about 30%) chance of being accepted by the bootstrap-style analysis. Even a 5,1,1 paper has a 1/27 chance of being accepted.

You’ve mentioned standing groups, and I’d definitely like there to be a standing BioNLP group of some kind, in my mind ideally a child of ICSB, but with some strong formal associations to ACL and AMIA.

-Alex Morgan

]]>