Task 14 Word Sense Induction & Disambiguation

V-Measure (VM)

System VM (All) VM (Verbs) VM (Nouns) Clusters Number Hermit 16.2 15.6 16.7 10.78 UoY 15.7 8.5 20.6 11.54 KSU KDD 15.7 12.4 18.0 17.5 Duluth-WSI 9.0 5.7 11.4 4.15 Duluth-WSI-SVD 9.0 5.7 11.4 4.15 Duluth-R-110 8.6 8.5 8.6 9.71 Duluth-WSI-Co 7.9 6.0 9.2 2.49 KCDC-PCGD 7.8 8.4 7.3 2.9 KCDC-PC 7.5 7.3 7.7 2.92 KCDC-PC-2 7.1 6.1 7.7 2.93 Duluth-Mix-Narrow-Gap 6.9 5.1 8.0 2.42 KCDC-GD-2 6.9 8.0 6.1 2.82 KCDC-GD 6.9 8.5 5.9 2.78 Duluth-Mix-Narrow-PK2 6.8 5.5 7.8 2.68 Duluth-MIX-PK2 5.6 5.2 5.8 2.66 Duluth-R-15 5.3 5.1 5.4 4.97 Duluth-WSI-Co-Gap 4.8 3.6 5.6 1.6 Random 4.4 4.6 4.2 4 Duluth-R-13 3.6 3.7 3.5 3 Duluth-WSI-Gap 3.1 1.5 4.2 1.4 Duluth-Mix-Gap 3.0 3.0 2.9 1.61 Duluth-Mix-Uni-PK2 2.4 4.7 0.8 2.04 Duluth-R-12 2.3 2.5 2.2 2 KCDC-PT 1.9 3.10 1.0 1.5 Duluth-Mix-Uni-Gap 1.4 3.0 0.2 1.39 KCDC-GDC 7.0 7.8 6.2 2.83 MFS 0.0 0.0 0.0 1.0 Duluth-WSI-SVD-Gap 0.0 0.1 0.0 1.02

Task 14 Word Sense Induction & Disambiguation

F-Score (FS)

System FS (All) FS (Verbs) FS (Nouns) Clusters Number MFS 63.4 72.7 57.0 1 Duluth-WSI-SVD-Gap 63.3 72.4 57.0 1.02 KCDC-PT 61.8 69.7 56.4 1.5 KCDC-GD 59.2 70.0 51.6 2.78 Duluth-Mix-Gap 59.1 65.8 54.5 1.61 Duluth-Mix-Uni-Gap 58.7 61.2 57.0 1.39 KCDC-GD-2 58.2 69.3 50.4 2.82 KCDC-GDC 57.3 70.0 48.5 2.83 Duluth-Mix-Uni-PK2 56.6 55.9 57.1 2.04 KCDC-PC 55.5 62.9 50.4 2.92 KCDC-PC-2 54.7 61.7 49.7 2.93 Duluth-WSI-Gap 53.7 53.9 53.4 1.4 KCDC-PCGD 53.3 65.6 44.8 2.9 Duluth-WSI-Co-Gap 52.6 51.5 53.3 1.6 Duluth-MIX-PK2 50.4 48.3 51.7 2.66 UoY 49.8 66.6 38.2 11.54 Duluth-Mix-Narrow-Gap 49.7 51.3 47.4 2.42 Duluth-WSI-Co 49.5 48.2 50.2 2.49 Duluth-Mix-Narrow-PK2 47.8 48.2 37.1 2.68 Duluth-R-12 47.8 52.6 44.3 2 Duluth-WSI-SVD 41.1 46.7 37.1 4.15 Duluth-WSI 41.1 46.7 37.1 4.15 Duluth-R-13 38.4 41.5 36.2 3 KSU KDD 36.9 54.7 24.6 17.5 Random 31.9 34.1 30.4 4 Duluth-R-15 27.6 28.9 26.7 4.97 Hermit 26.7 30.1 24.4 10.78 Duluth-R-110 16.1 16.4 15.8 9.71

Task 14 Word Sense Induction & Disambiguation

Supervised Recall (SR), test set split:80% mapping, 20% evaluation

System SR (All) SR (Nouns) SR (Verbs) UoY 62.44 59.43 66.82 Duluth-WSI 60.46 54.66 68.92 Duluth-WSI-SVD 60.46 54.66 68.92 Duluth-WSI-Co-Gap 60.34 54.09 68.65 Duluth-WSI-Co 60.27 54.68 67.6 Duluth-WSI-Gap 59.81 54.36 67.76 KCDC-PC-2 59.76 54.09 68.04 KCDC-PC 59.73 54.55 67.29 KCDC-PCGD 59.53 53.33 68.56 KCDC-GDC 59.08 53.39 67.38 KCDC-GD 59.03 52.97 67.87 KCDC-PT 58.88 53.07 67.35 KCDC-GD-2 58.72 52.78 67.38 Duluth-WSI-SVD-Gap 58.69 53.22 66.66 MFS 58.67 53.22 66.629 Duluth-R-12 58.46 53.05 66.44 Hermit 58.34 53.56 65.30 Duluth-R-13 58.01 52.27 66.38 Random 57.25 51.45 65.69 Duluth-R-15 56.76 50.91 65.3 Duluth-Mix-Narrow-Gap 56.63 48.11 69.06 Duluth-Mix-Narrow-PK2 56.15 47.54 68.7 Duluth-R-110 54.75 48.28 64.2 KSU KDD 52.18 46.63 60.28 Duluth-MIX-PK2 51.62 41.1 66.96 Duluth-Mix-Gap 50.61 40.04 66.02 Duluth-Mix-Uni-PK2 19.29 1.82 44.78 Duluth-Mix-Uni-Gap 18.72 1.55 43.76

Task 14 Word Sense Induction & Disambiguation

Supervised Recall (SR), test set split:60% mapping, 40% evaluation

System SR (All) SR (Nouns) SR (Verbs) UoY 61.96 58.62 66.82 Duluth-WSI-Co 60.07 54.59 68.05 Duluth-WSI-Co-Gap 59.51 53.45 68.33 Duluth-WSI-SVD 59.48 53.45 68.27 Duluth-WSI 59.48 53.45 68.27 Duluth-WSI-Gap 59.32 53.19 68.23 KCDC-PCGD 59.10 52.60 68.56 KCDC-PC-2 58.90 53.35 66.99 KCDC-PC 58.89 53.58 66.64 KCDC-GDC 58.29 52.14 67.26 KCDC-GD 58.27 51.88 67.59 MFS 58.25 52.45 66.70 KCDC-PT 58.25 52.18 67.11 Duluth-WSI-SVD-Gap 58.24 52.45 66.66 KCDC-GD-2 57.90 51.67 66.99 Duluth-R-12 57.72 51.74 66.42 Duluth-R-13 57.59 51.13 67.00 Hermit 57.27 52.53 64.16 Duluth-R-15 56.53 49.95 66.11 Random 56.52 50.21 65.73 Duluth-Mix-Narrow-Gap 56.19 47.67 68.59 Duluth-Mix-Narrow-PK2 55.65 46.86 68.45 Duluth-R-110 53.60 46.70 63.63 Duluth-MIX-PK2 50.46 39.70 66.13 KSU KDD 50.42 44.25 59.41 Duluth-Mix-Gap 49.77 38.86 65.64 Duluth-Mix-Uni-PK2 19.12 1.77 44.40 Duluth-Mix-Uni-Gap 18.91 1.52 44.23