A Swedish COVID-19 (sv-COVID-19) corpus and its exploration … smorgasbord

As the COVID-19 virus became a pandemic in March 2020, the amount of (time-stamped written) data, such as news/newspaper reports, scientific articles, social media posts (e.g. blogs and twitter), surveys and other information about the virus and its symptoms, prevention, management and transmission became massively available. Such data contained both valid and reliable information, and relevant facts from trusted sources and also rumors, conspiracy theories and misinformation from unofficial ones. However, it was not only the amount of (written) data and information …

The SwedishGLUE project

Artificial intelligence system dealing with (human) natural language rely on language models, predictions of which words occur together. To better understand how such models work — and where they fail — when applied to Swedish texts we need Swedish test data. A collection of test data addressing various aspects of understanding and generating text allows us to evaluate and compare models. During the autumn of 2020 we have started working on developing evaluation data for Swedish language models at Språkbanken Text. This …

How native and non-native speakers talk to each other

We at Språkbanken Text have just released a new corpus of native (L1) and non-native (L2) speech in four languages: English, Spanish, French and Italian. The corpus contains more than 170 million words produced by more than 97 thousand speakers (size varies a lot across the four languages, though). The corpus has been created by scraping WordReference forums, where users discuss various questions about languages. Importantly, every user has to provide their native language, and this information, alongside with the nickname, is …

Flerordingar: ord som består av flera delar

När vi tänker på ord så tänker vi oftast på enheter som i text omges av blanksteg (mellanrum): ’huset’, ’superstor’, ’bloggade’. De flesta skulle nog säga att ’idag’ är ett ord, men hur är det om vi skriver det (också rättstavat) ’i dag’ då? ’Mont Blanc-tunneln’? ’Röda blodkroppar’? I det här blogginlägget tänkte jag prata om ord som innehåller mellanrum och flerordsuttryck, och hur man kan analysera dem i en korpus. Om vi ska annotera en text, alltså märka upp den med …

How reliable is sense disambiguation in texts by native and non-native speakers?

(This blog is based on a joint research and publication in collaboration with David Alfter, Therese Lindström Tiedemann, Maisa Lauriala and Daniela Piipponen) At our department, and outside, we are used to search Korp corpora using the linguistic categories available there. Some of us know that these linguistic categories come as a result of automatic annotation by the Sparv-pipeline. The pipeline automatically splits raw text into tokens, sentences, finds a base form to each of the running (inflected) words, assigns word classes, …

Grierson’s “Linguistic Survey of India” as open-access digital data resource for studying languages of South Asia

Lars Borin, Anju Saxena, Shafqat Mumtaz Virk, Bernard Comrie South Asia – comprising the seven countries Pakistan, India, Nepal, Bhutan, Bangladesh, Sri Lanka, and the Maldives, as well as immediately adjacent areas of neighboring countries (parts of Afghanistan, China, and Myanmar) – is the home of hundreds of languages belonging to several unrelated language families. The region has a long history of far-ranging multilingualism and close linguistic and cultural contacts, the details of which are still far from completely understood. Today, the …

En syntaktisk beskrivningsmodell för modern svensk text

Sverige har en relativt lång tradition av att skapa en typ av korpus som brukar kallas trädbank. En trädbank är en samling texter som har annoterats (märkts upp) med ordklasser och syntaktisk struktur. Den syntaktiska strukturen för en mening kan ritas upp så att den liknar ett träd. Trädbanken Talbanken skapades redan på 70-talet (Teleman, 1974) och texterna (och delar av annoteringen) har återanvänts i flera trädbanker sedan dess. Trädbankerna kan sedan t ex användas för att studera grammatiska frågor, för att …

The five lives of Talbanken

A dependency tree from Talbanken

This post is about Talbanken, one of the most widely used and important Swedish corpora. There exist at least five versions of this treebank, and the purpose of this post is to reduce ambiguity of the name ”Talbanken”, which sometimes leads to confusion. I am going to list the five versions, explain the basic differences between them and suggest unambiguous version names.

Om ordklasser för svenska språket

Ordklassindelning används i många språkteknologiska verktyg därför att det är ett sätt att skilja mellan olika användningar av ett ord. Genom ordklasserna kan man enklare söka efter liknande ord och uttryck i stora textmängder, eller skapa en ny text med liknande form. Att automatiskt dela in orden i texter i ordklasser är därmed en av de grundläggande metoderna inom artificiell intelligens, för att få datorn att förstå mänskligt språk. Människan har länge delat in ord i olika klasser eller kategorier, beroende på …

Argumentation Mining

What if you could find all arguments in a text without having to read it? Or, what if you could search a database for a controversial topic and immediately get arguments for and against it, gathered from text all around the internet? Or, imagine when writing an essay you would automatically get an estimation of how persuasive your arguments are. Scenarios such as these could be possible with techniques developed in the field of argumentation mining. The aim of this relatively new …