Skip to main content

SemEval 2020 Task 1 (repackaged for SuperLim)

Swedish Test Data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection
I. IDENTIFYING INFORMATION
Title* SemEval 2020 Task 1
Subtitle Swedish Test Data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection (repackaged for SuperLim)
Created by* Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky and Nina Tahmasebi
Publisher(s)* Språkbanken Text (sb-info@svenska.gu.se)
Link(s) / permanent identifier(s)* Resource page in the SuperLim collection, containing the SuperLim-style documentation (this document): https://spraakbanken.gu.se/en/resources/semeval2020t1-superlim. Original resource page, containing the data: https://spraakbanken.gu.se/en/resources/semeval2020
License(s)* CC BY 4.0
Abstract* This data collection contains the Swedish test data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection: a Swedish text corpus pair (one corpus covering the time period 1790-1830, another 1895-1903) and 31 lemmas which have been annotated for their lexical semantic change between the two corpora.
Funded by* Swedish Research Council (dnr 2018-01184)
Cite as [1], [2]
Related datasets Part of the SuperLim collection (https://spraakbanken.gu.se/en/resources/superlim).
II. USAGE
Key applications Machine Learning, Semantic change detection, Evaluation of language models
Intended task(s)/usage(s) Evaluate models on the following task: (1) given two the corpora (for time periods t1 and t2) and the set of target words, decide which words lost or gained senses between t1 and t2, and which ones did not; as annotated by human judges; (2) given the two corpora, rank the set of target words according to their degree of lexical semantic change between t1 and t2, as annotated by human judges. A higher rank means stronger change.
Recommended evaluation measures (1) Accuracy; (2) Spearman's correlation coefficient (see [1] for a possible strategy of dealing with ties)
Dataset function(s) Testing
Recommended split(s) Test data only
III. DATA
Primary data* Text
Language* Swedish
Dataset in numbers* 31 target words; two corpora (71 million tokens for 1790-1830; 111 million tokens 1895-1903).
Nature of the content* The list of target words and two corpora
Format* Corpora: plain text. Target word lists: two columns (word and label).
Data source(s)* See [1]
Data collection method(s)* See [1]
Data selection and filtering* See [1]
Data preprocessing* See [1]
Data labeling* See [1]
Annotator characteristics See [1]
IV. ETHICS AND CAVEATS
Ethical considerations
Things to watch out for
V. ABOUT DOCUMENTATION
Data last updated* 2020-02-19, v1.0
Which changes have been made, compared to the previous version* This is the first official version
Access to previous versions
This document created* 2021-06-15, Aleksandrs Berdicevskis
This document last updated* 2021-06-15, Aleksandrs Berdicevskis
Where to look for further details [1], [2], [3]
Documentation template version* v1.0
VI. OTHER
Related projects
References [1] Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky and Nina Tahmasebi.
SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection. In Proceedings of the 14th International Workshop on Semantic Evaluation, Barcelona, Spain, 2020. Association for Computational Linguistics https://www.aclweb.org/anthology/2020.semeval-1.1/
[2] Swedish Test Data for SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection, https://zenodo.org/record/3730550#.YMiazfkzZPY
[3] SemEval 2020 Task 1: Unsupervised Lexical Semantic Change Detection https://competitions.codalab.org/competitions/20948#learn_the_details-d…

Contact

Språkbanken (sb-info@svenska.gu.se)