SweDiagnostics

Swedish version of the SuperGLUE diagnostic dataset.

Manual translation of the SuperGLUE Diagnostic Dataset. The data includes all annotated original sentence pairs of SuperGLUE and their Swedish translations.

License: Creative Commons CC-BY 4.0 International (please refer to Språkbanken, University of Gothenburg, Sweden).

  <td>I. IDENTIFYING INFORMATION</td>
  <td></td>
</tr>
<tr>

  <td>Title*</td>
  <td>SweDiagnostics</td>
</tr>
<tr>

  <td>Subtitle</td>
  <td></td>
</tr>
<tr>

  <td>Created by*</td>
  <td>Felix Morger, Gothenburg University (felix.morger@gu.se)</td>
</tr>
<tr>

  <td>Publisher(s)*</td>
  <td>Språkbanken Text (sb-info@svenska.gu.se)</td>
</tr>
<tr>

  <td>Link(s) / permanent identifier(s)*</td>
  <td>https://spraakbanken.gu.se/en/resources/superlim</td>
</tr>
<tr>

  <td>License(s)*</td>
  <td>CC BY 4.0</td>
</tr>
<tr>

  <td>Abstract*</td>
  <td>Manual Swedish translation of all 1106 sentence pairs of the SuperGLUE diagnostic dataset.</td>
</tr>
<tr>

  <td>Funded by*</td>
  <td>Vinnova (grant no. 2020-02523)</td>
</tr>
<tr>

  <td>Cite as</td>
  <td></td>
</tr>
<tr>

  <td>Related datasets</td>
  <td>SuperLim, SuperGLUE diagnostic dataset, FraCaS test suite</td>
</tr>
<tr>

  <td></td>
  <td></td>
</tr>
<tr>

  <td>II. USAGE</td>
  <td></td>
</tr>
<tr>

  <td>Key applications</td>
  <td>Fine-grained analysis of system performance on a broad range of linguistic phenomena.</td>
</tr>
<tr>

  <td>Intended task(s)/usage(s)</td>
  <td>Natural language inference.</td>
</tr>
<tr>

  <td>Recommended evaluation measures</td>
  <td>Matthew's correlation coefficient.</td>
</tr>
<tr>

  <td>Dataset function(s)</td>
  <td></td>
</tr>
<tr>

  <td>Recommended split(s)</td>
  <td>No split.</td>
</tr>
<tr>

  <td></td>
  <td></td>
</tr>
<tr>

  <td>III. DATA</td>
  <td></td>
</tr>
<tr>

  <td>Primary data*</td>
  <td>Text</td>
</tr>
<tr>

  <td>Language*</td>
  <td>Swedish</td>
</tr>
<tr>

  <td>Dataset in numbers*</td>
  <td>1106</td>
</tr>
<tr>

  <td>Nature of the content*</td>
  <td>Pairs of sentences annotated according with their inference relation and the linguistic phenomena that account for their differencs</td>
</tr>
<tr>

  <td>Format*</td>
  <td>Comma-separated</td>
</tr>
<tr>

  <td>Data source(s)*</td>
  <td>SuperGLUE Diagnostic Dataset: Pruksachatkun, Yada &amp; Nangia, Nikita &amp; Singh, Amanpreet &amp; Michael, Julian &amp; Hill, Felix &amp; Levy, Omer &amp; Bowman, Samuel. (2019). SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.</td>
</tr>
<tr>

  <td>Data collection method(s)*</td>
  <td>See original source.</td>
</tr>
<tr>

  <td>Data selection and filtering*</td>
  <td>See original source.</td>
</tr>
<tr>

  <td>Data preprocessing*</td>
  <td>See original source.</td>
</tr>
<tr>

  <td>Data labeling*</td>
  <td>Some data labels (annotations) were changed to fit with Swedish example, but in general the aim was to keep such changes to a minimum.</td>
</tr>
<tr>

  <td>Annotator characteristics</td>
  <td></td>
</tr>
<tr>

  <td></td>
  <td></td>
</tr>
<tr>

  <td>IV. ETHICS AND CAVEATS</td>
  <td></td>
</tr>
<tr>

  <td>Ethical considerations</td>
  <td>See original data source.</td>
</tr>
<tr>

  <td>Things to watch out for</td>
  <td>See original data source.</td>
</tr>
<tr>

  <td></td>
  <td></td>
</tr>
<tr>

  <td>V. ABOUT DOCUMENTATION</td>
  <td></td>
</tr>
<tr>

  <td>Data last updated*</td>
  <td>2021-06-04, v1.0</td>
</tr>
<tr>

  <td>Which changes have been made, compared to the previous version*</td>
  <td>Full translation coverage.</td>
</tr>
<tr>

  <td>Access to previous versions</td>
  <td></td>
</tr>
<tr>

  <td>This document created*</td>
  <td>2021-06-04, Felix Morger.</td>
</tr>
<tr>

  <td>This document last updated*</td>
  <td>2021-06-04, Felix Morger.</td>
</tr>
<tr>

  <td>Where to look for further details</td>
  <td></td>
</tr>
<tr>

  <td>Documentation template version*</td>
  <td>1</td>
</tr>
<tr>

  <td></td>
  <td></td>
</tr>
<tr>

  <td>VI. OTHER</td>
  <td></td>
</tr>
<tr>

  <td>Related projects</td>
  <td></td>
</tr>
<tr>

  <td></td>
  <td></td>
</tr>
<tr>

  <td>References</td>
  <td></td>
</tr>