A Swedish COVID-19 (sv-COVID-19) corpus and its exploration … smorgasbord

As the COVID-19 virus became a pandemic in March 2020, the amount of (time-stamped written) data, such as news/newspaper reports, scientific articles, social media posts (e.g. blogs and twitter), surveys and other information about the virus and its symptoms, prevention, management and transmission became massively available. Such data contained both valid and reliable information, and relevant facts from trusted sources and also rumors, conspiracy theories and misinformation from unofficial ones. However, it was not only the amount of (written) data and information …

The SwedishGLUE project

Artificial intelligence system dealing with (human) natural language rely on language models, predictions of which words occur together. To better understand how such models work — and where they fail — when applied to Swedish texts we need Swedish test data. A collection of test data addressing various aspects of understanding and generating text allows us to evaluate and compare models. During the autumn of 2020 we have started working on developing evaluation data for Swedish language models at Språkbanken Text. This …

Reflektioner från SLTC 2020

Humanister exteriör

25-27 november gick den åttonde upplagan av SLTC, Swedish Language Technology Conference, av stapeln på Humanisten här i Göteborg. Eller, skulle ha gjort om inte ett visst virus satte stopp för det. Istället fick vi som alla andra ställa om till en helt digital utgåva, men det funkade det med. Vi fick ett rekord i antalet registreringar: 193 deltagare från 34 olika länder! (Majoriteten, 60%, kom dock från Sverige). Inte alla dök förstås upp – dels var registreringen gratis, och dels var …

How native and non-native speakers talk to each other

We at Språkbanken Text have just released a new corpus of native (L1) and non-native (L2) speech in four languages: English, Spanish, French and Italian. The corpus contains more than 170 million words produced by more than 97 thousand speakers (size varies a lot across the four languages, though). The corpus has been created by scraping WordReference forums, where users discuss various questions about languages. Importantly, every user has to provide their native language, and this information, alongside with the nickname, is …

Pseudonymization of learner essays as a way to meet GDPR requirements

This blog is based on the author’s (Elena Volodina’s) joint research with Yousuf (Samir) Ali Mohammed, Arild Matsson, Beáta Megyesi and Sandra Derbring Access to language data is an obvious prerequisite for research in digital humanities in general, and for the development of NLP-based tools in particular. However, accessible data becomes a challenging target where personal data is involved. This is very true of language learner data where tasks are often phrased so that they, directly or indirectly, elicit explicit personal information, …

Flerordingar: ord som består av flera delar

När vi tänker på ord så tänker vi oftast på enheter som i text omges av blanksteg (mellanrum): ’huset’, ’superstor’, ’bloggade’. De flesta skulle nog säga att ’idag’ är ett ord, men hur är det om vi skriver det (också rättstavat) ’i dag’ då? ’Mont Blanc-tunneln’? ’Röda blodkroppar’? I det här blogginlägget tänkte jag prata om ord som innehåller mellanrum och flerordsuttryck, och hur man kan analysera dem i en korpus. Om vi ska annotera en text, alltså märka upp den med …

How reliable is sense disambiguation in texts by native and non-native speakers?

(This blog is based on a joint research and publication in collaboration with David Alfter, Therese Lindström Tiedemann, Maisa Lauriala and Daniela Piipponen) At our department, and outside, we are used to search Korp corpora using the linguistic categories available there. Some of us know that these linguistic categories come as a result of automatic annotation by the Sparv-pipeline. The pipeline automatically splits raw text into tokens, sentences, finds a base form to each of the running (inflected) words, assigns word classes, …

Grierson’s “Linguistic Survey of India” as open-access digital data resource for studying languages of South Asia

Lars Borin, Anju Saxena, Shafqat Mumtaz Virk, Bernard Comrie South Asia – comprising the seven countries Pakistan, India, Nepal, Bhutan, Bangladesh, Sri Lanka, and the Maldives, as well as immediately adjacent areas of neighboring countries (parts of Afghanistan, China, and Myanmar) – is the home of hundreds of languages belonging to several unrelated language families. The region has a long history of far-ranging multilingualism and close linguistic and cultural contacts, the details of which are still far from completely understood. Today, the …

En syntaktisk beskrivningsmodell för modern svensk text

Sverige har en relativt lång tradition av att skapa en typ av korpus som brukar kallas trädbank. En trädbank är en samling texter som har annoterats (märkts upp) med ordklasser och syntaktisk struktur. Den syntaktiska strukturen för en mening kan ritas upp så att den liknar ett träd. Trädbanken Talbanken skapades redan på 70-talet (Teleman, 1974) och texterna (och delar av annoteringen) har återanvänts i flera trädbanker sedan dess. Trädbankerna kan sedan t ex användas för att studera grammatiska frågor, för att …

Korp searches in Second Language data

Korp offers a lot of different corpus collections for various types of search (and research). Swedish as a Second Language (L2) is one of the subcategories of the language that can be studied with the help of Korp. At the moment, Korp provides access to five L2 corpora through its interface: ASU – Andraspråksutveckling SpIn – texts from the centrum for Språkintroduktion SW1203 – texts from a preparatory course for university students SweLL – Swedish Learner Language – adult-written essays from a …