The Senseval Committee invites proposals for tasks to be run as part of SemEval-2010.  Following the spirit of the past SemEval-2007,  the nature of the tasks in previous Senseval editions has evolved to include semantic analysis tasks outside of word sense disambiguation.

Thus we welcome proposals for any tasks that can test an automatic system for semantic analysis of text, be it application dependent or independent. We especially encourage tasks for different languages, cross-lingual tasks, and tasks that are relevant to particular NLP applications such as machine translation, information retrieval and information extraction.

It will help us if tasks are designed in such that they can be run and scored automatically from a central website (as in Senseval-3 and SemEval-2007). We'll be happy to help with task design, data formatting, and so on.

Information on the design of previous Senseval-1/2/3 tasks and SemEval-2007 can be found at:

http://nlp.cs.swarthmore.edu/semeval/index.php

http://www.senseval.org/senseval3/

http://193.133.140.102/senseval2/

http://www.itri.brighton.ac.uk/events/senseval/ARCHIVE/index.html

The time period for SemEval-2010 has not yet been finalised, but it will be held over a 1-2 month period in the first part of 2010.

SUBMISSION DETAILS

Proposals for tasks will ideally contain:

- A description of the task (about 1 page)

- How the training/testing data will be built and/or procured

- The evaluation methodology to be used including clear evaluation criteria

- The availability of the resources to the participants (copyright, costs, etc)

- The resources required to prepare the task (time, money, and human)

If you are not yet at a point to provide outlines of all of these, that's fine, but please give some thought to each, and present an overview of your first ideas.  We will gladly give feedback.

Please submit proposals as soon as possible, preferably by electronic mail in plain ASCII text to the SemEval-2010 chairs:

Katrin Erk, University of Texas at Austin (katrin.erk at mail.utexas.edu)

Carlo Strapparava, FBK-irst, Italy (strappa at fbk.eu)

IMPORTANT DATES

Sep 21, 2008  Intention to submit (send a short-paragraph description)

Oct 19, 2008  Submission deadline for task proposals

Nov 16, 2008  Notification of acceptance

In SemEval-2010 we are going to give plenty of time both in tasks preparation phase, especially regarding the tuning of data sets, and to task participants to prepare for the task and to tune their systems. Along this rationale, we strongly recommend that each task organizers come up with the presentation of trial data set,  well in advance (roughly, summer 2009). Please consider this point in designing the task schedule.