Background
The Vinnova-funded project SuperLim has developed a first test collection to evaluate Swedish language models. The test collection has been developed based on well-established and scientifically evaluated English collections such as (Super)GLUE, and has generated a dozen separate test sets that are available via Språkbanken´s website. However, the scope of SuperLim was limited, and three additional components are missing to be able to realize the full value of a test collection.
Project description
The purpose of the present project SuperLim 2.0 is to contribute these additional components. SuperLim 2.0 complements SuperLim 1.0 with training data, which is needed to create models under the same conditions, thereby enabling accurate comparisons between models. SuperLim 2.0 has a reference implementation with results, which can serve as a baseline for being able to compare models with each other and improve them. SuperLim 2.0 has a leaderboard where you can publish results, compare with baseline, and follow the development of the Swedish language models.
Participating institutions
Språkbanken Text, AI Sweden, National Library of Sweden (KB-Lab), RISE.
