Mini-workshop
Date: December, 16, 2024
Time: 13.15-15.00
Room: J415 (Humanisten)
Program
13.15 -- Arianna Masciolini and Ricardo Muñoz Sánchez, University of Gothenburg, Sweden
MultiGEC-2025 – a Multilingual Grammatical Error Correction shared task
Abstract: In this talk, we will give an overview of MultiGEC-2025, the first text-level multilingual shared task in Grammatical Error Correction. The dataset assembled for the shared task covers 12 European languages and several domains. We will also discuss the motivation behind the initiative, as well as our design choices for the shared task and its results.
13.40 -- Andrew Caines, Cambridge University, United Kingdom
The design of the Write & Improve Corpus 2024 and multilingual evaluation with ERRANT for MultiGEC-2025
Abstract: I will give an overview of the design and preparation of the Write & Improve Corpus 2024, which was the English component of the MultiGEC-2025 shared task. I will also explain how we adapted ERRANT for multilingual evaluation in MultiGEC-2025. This was essential for obtaining precision, recall and F0.5 scores for all 12 languages in the shared task.
14.00 -- Orphée De Clercq and Joni Kruijsbergen, Ghent University, Belgium
Automated Writing Support for Dutch
Abstract: Large Language Models have had a clear impact on the field of Automated Writing Support (AWS), not in the least on the tasks of Grammatical Error Detection (GED) & Correction (GEC). Our work targets writing support for L1 and L2 Dutch speakers. Last year, we explored GED for Dutch by comparing two distinct approaches: fine-tuning BERT-models and zero-shot with proprietary GPT. Within the first setting, we evaluated several mono- and multilingual models to assess their effectiveness. In this workshop, we will share our findings across those experimental settings, highlight the challenges we encountered and discuss how we are currently addressing them.
14.20 Murathan Kufrfali, Stockholm university, Sweden
Evaluating models for Grammatical Error Correction
Abstract: Traditional evaluation methods for Grammatical Error Correction (GEC) fail to fully capture the full range of system capabilities and objectives. The emergence of large language models (LLMs) has further highlighted the shortcomings of these evaluation strategies, emphasizing the need for a paradigm shift in evaluation methodology. In this talk I will present a comprehensive evaluation of various GEC systems using a recently published dataset of Swedish learner texts. We suggest using human post-editing of GEC system outputs to analyze the amount of change required to reach native-level human performance on the task, and provide a dataset annotated with human post-edits and assessments of grammaticality, fluency and meaning preservation of GEC system outputs.
14.40 -- Open discussion
Moderators: Elena Volodina and Julia Prentice