Evaluation Exercises on Semantic Evaluation - ACL SigLex event
#6 Classification of Semantic Relations between MeSH Entities in Swedish Medical Texts
There is a growing interest and, consequently, a volume of publications related to the topic of relation classification in the medical domain. Algorithms for classifying semantic relations have potential applications in many language technology applications and there has been a renewed interest during the last years. If such semantic relations can be determined, the potential of obtaining more accurate results for systems and applications such as Information Retrieval and Extraction, Summarization, Question Answering, etc. increases, particularly since searching to mere co-occurrence of terms is unfocused and does not by any means guarantee that there can be a relation between the identified terms of interest. For instance, knowing the relationship that prevails between a medication and a disease or symptom should be useful for searching free text and easier obtaining answers to questions such as “What is the effect of treatment with substance X to the disease Y?”,
Our task "Classification of Semantic Relations between MeSH Entities in Swedish Medical Texts" deals with the classification of semantic relations between pairs of MeSH entities/annotations. We focus on three entity types: DISEASES/SYMPTOMS (category C in the MeSH hierarchy), CHEMICAL and DRUGS/ANALYTICAL, DIAGNOSTIC AND THERAPEUTIC TECHNIQUES AND EQUIPMENT (categories D and E in the MeSH hierarchy). The evaluation task is similar to the SEMEVAL-1/Task#4 by Girju et al.: Classification of Semantic Relations between Nominals. This implies that the evaluation methodology to be used will include similar evaluation criteria already developed (the SEMEVAL-1/Task#4).
The datasets for the task will consist of annotated sentences with relevant MeSH entities, including the surrounding context for the investigated entities and their relation within a window size of one to two preceding and one to two following sentences. We plan to have about nine semantic relations with approx. 100-200 training sentences and 50-100 testing sentences per relation.