HOME   CALL FOR TASKS CALL FOR INTEREST Workshop Submission Workshop PROGRAM NEWS   REGISTRATION TASKS (short list) TASKS (with rankings) DATA   [41 available]
LRE special issue  


Evaluation Exercises on Semantic Evaluation - ACL SigLex event



Language Resources and Evaluation journal

Special Issue on Computational Semantic Analysis of Language: SemEval-2010

Note: Submission Deadline Extended to April 8, 2011


SemEval-2010, an ACL SigLex event, hosted 16 shared tasks for evaluating semantic analysis systems. Over 130 teams participated with over than 230 submitted systems.

It took place in July 15-16 2010, as a co-located event with the 48th Annual Meeting of the ACL, held in Uppsala, Sweden. It was the fifth international evaluation exercise following the series of successful Senseval/SemEval workshops. Senseval-1 (1998), Senseval-2 (2001), Senseval-3 (2004) were more oriented toward word sense disambiguation tasks. For the fourth edition, the name was changed to SemEval-2007, motivated by a desire to broaden the spectrum of tasks to all aspects of computational semantic analysis of language. SemEval-2010 continued in this vein, with some continuations of existing tasks but also many new tasks for a wider variety of semantic phenomena.


We invite submissions of papers describing in-depth evaluation on computational semantics of text. A natural candidate is an extended "task description" paper from SemEval-2010 (especially the novel tasks), which includes a broad description of the problem, resources used, system comparative, analysis of results, and some novel conclusions.

But we do not limit the contributions to SemEval-related works. Instead, other works presenting substantial experimental evaluation on natural language semantics are also welcome. Finally, papers describing concrete NLP systems/approaches on one of those semantic tasks are welcome too, provided they present relevant evaluation on the task at hand and highlight important properties of the problem, resources, etc. System description papers which don't add insights on the specific problem they tackled, as well descriptions of already known and established tasks, are out of the scope of this issue.

The list of topics of interest includes but is not limited to:

  • Cross-lingual and multilingual approaches to semantic analysis
  • Domain adaptation
  • Deeper semantic analysis: tasks and approaches
  • Grounding meaning in extralinguistic knowledge
  • Different approaches for describing word meaning
  • The inclusion of discourse context in semantic analysis
  • Automatic recognition of semantic relations


  • Call for papers: 15 November 2010
  • Submissions of papers (extended): 31 March 2011 8 April 2011
  • Preliminary decisions to authors: July 2011 (preliminary)
  • Submission of revised articles: August 2011 (preliminary)
  • Final decisions to authors: October 2011 (preliminary)
  • Final versions due from authors: November 2011 (preliminary)


To prepare the papers, please follow the style guidelines provided by the LRE journal

To submit papers:



Updated information on the Special Issue at: http://semeval2.fbk.eu

© 2008 FBK-irst  |  internal area