Skip to main content

WG1 workshop, Gothenburg (Sweden), 5-7 Dec 2018

Learning materials through crowdsourcing: teachers, perspectives & scenarios

Venue: University of Gothenburg, Lennart Torstenssongata 6, K332 / K333

Maps and directions

  • From the airport: if you take a Flight bus (Swe "flygbussen") and go to the Hotel Panorama or to the venue (GU, Lennart Torstenssons gata 8), you should get off at Korsvägen - the first stop.
  • Map Korsvägen - Hotel Panorama. Make sure the Gothia Tower (Svenska Mässan) buildings are behind you.
  • Map Korsvägen - Lennart Torstenssons gata 8. Pay attention that the map leads you to Lennart Torstenssonsgata 4. From there, enter a parking lot through a (pretty narrow) stone fence opening, cross the parking and go up the stairs to the building. We will (try not to forget to) put signs leading you to the meeting room.
  • Map Hotel Panorama - Lennart Torstenssonsgata 8. Note that the turn from Eklandagata on the map leads you to a long staircase that seemingly leads you to a construction site. You go through the park area, and come to the alley where the building for our meeting is (marked as "Work" on the map). We will try to set signs leading you to the meeting room from all possible entry points.
    If you think this is a scary way - go through Korsvägen instead. It will take 10 more minutes.

General information

Work Group 1 of enet-COLLECT invites WG1 members (and if funding allows, people outside WG1) to join a hands-on workshop on the topic of "Learning materials through crowdsourcing: teachers, perspectives & scenarios" organized in Gothenburg, Sweden, 5-7 December 2018.

This WG1 meeting is aimed at providing a hands-on exercise for the group members. We will test a small-scale crowdsourcing experiment together, as well as introduce all steps from the beginning to the end involved in the process of setting a crowdsourcing experiment. To make it manageable, we narrow the topic to multi-word expressions in second language learning context.

Before the meeting, the participants will be asked to complete a crowdsourcing (mini-)task based on English data.

The program contains presentations, discussions, individual and group work.

Registration form

Registration is now closed. (To register for the workshop, please enter information into this form. There are quite specific questions there, so count that it might take some time to fill in the form.

NOTE! We need your registration by August, 24, to secure fair candidate ranking and good flight ticket prices, with a potential to expand the number of participants.)

Selection of participants

Participants have been selected from among the registered ones (see section above), based on the letter of interest (max 1000 characters) written in a free form directly in the registration form. (e.g. your interest in this exercise, motivation, relation between your work and crowdsourcing, aim of learning these aspects, availability for a group-wise experiment, etc.)

Members of WG1, and especially WG1 members actively participating in the WG1 tasks, were given priority. The final number of approved/financed participants is 28, and some local people are expected to join us.

Reimbursement principles

Several conditions should be met

  • Participation in the pre-workshop crowdsourcing task
  • Literature review: 3 articles (instructions will be provided later)


Time Program point.
  Day 1: December, 5th Chair: Iztok Kosem
13.00-14.00 Lunch at the venue (Lennart Torstenssongata 8, room K332)
14.00-15.30 Presentations:
Elena Volodina Welcome & introduction [slides]
Elena Volodina Introduction to the pre-workshop MWE experiment [slides]
  • For testing purposes:
  • Link to adverbial MWE experiment
  • Link to verbal MWE experiment
  • Link to guidelines
Jaka Čibej & David Alfter Experiement set up and results of the MWE experiment [Slides Jaka] [Slides David]
15.30-16.00 Coffee break
16.00-18.00 Discussion: Together, in groups and individually:
experiences of being a "crowd", brainstorming ideas for further scenarios, sketching individual (or small-group) plans
You will also be given a chance to browse and explore the data from our MWE experiment
  • Group divisions [list]
  • Feedback form [link]
  • Discussion questions 1 [link]
  • Discussion questions 2 (in small groups) [link]
  • Linear scales (OpenOffice file) [link]
  • Linear scales (Excel file) [link
  • Adverbial MWE embedding [link]
  • Verbal MWE embedding [link]
19.00-... Dinner -- Panorama Hotel. Address: Eklandagatan 51, Gothenburg
  Day 2: December, 6th
  Chair: Elena Volodina
09.00-09.05 Introduction and organization announcements
09.05-09.45 Liam Murray and Marta Giralt The Digilanguages portal: teacher motivation, crowd sizing in crowd sourcing, implementation, validation and sustainability [Slides]
09.45-10.30 Spela Arhar Holdt Teacher survey on integration of and attitudes to crowdsourcing [slides]
10.30-11.00 Coffee break
11.00-11.45 Discussion: Rina Zviel Girshin and Elizabeth Gajek: Implications of the teacher survey, experiences as teachers and with teachers [slides]
11.45-12.30 Lars Borin: False truths and true falsehoods: Typological linguistics and multi-word expressions [Slides]
12.30-14.00 Lunch -- Restaurant Tabla, at World Culture Museum. Address: Södra vägen 54, Gothenburg
  Chair: David Alfter
14.00-14.40 pyBossa introduction
Jaka Čibej: a bit of technical insight, data format, data upload, requirements, etc. + hands-on exercise [slides]
14.40-15.30 Hands-on session: use the pre-processed data to set up a (controlled) type of crowdsourcing task; import data, visualize
  PyBossa test sheet
  PyBossa task presenter
15.30-16.00 Coffee break
16.00-16.20 Iztok Kosem Different approaches to crowdsourcing collocations. [Slides]
16.20-16.40 Peter Dekker & Tanneke Schoonheim When to use PYBOSSA? Case studies on crowdsourcing for Dutch. [Slides]
  Chair: Iztok Kosem
16.45-18.00 Pitching ideas (a lá "enet-Collect's got talent"): revise ideas from day1, structure acc to the flow, hand-in
  Crowdsourcing idea description template [Link]
18.30-... Dinner -- Pub quiz at Foxes. Address: Gibraltargatan 8, Gothenburg
Max 3 people in a team. Food will need to be pre-ordered before 18.00 [menu] [translation of the menu] , since the quiz itself starts at 19.00. The quiz will be in English (they promised), though I have no idea what the topic will be. Hopefully not entirely focused on the Swedish culture )))
  Day 3: December, 7th Chair: Jaka Čibej
9.00-10.30 Presentations and discussions: ideas from the day before. Any potential for cross-lingual experiment? Which idea? Data need? Data availability? Expertise?
10.30-11.00 Coffee break
11.00-11.30 Elena Volodina Summary: what we have learnt, awareness of limitations, benefits, reality checks, etc. [Slides]
11.30-12.30 Other matters:
David Alfter Literature review: status and needs. Hands-on practice [Zotero library] [Literature review guidelines]
Elena Volodina Planning ahead, closing [slides]
12.30-13.30 Lunch -- Berzelius bar. Address: Södra Vägen 20, 412 54 Gothenburg
17.00-19.30 Dinner Moon Thai Kitchen. Address: Kristinelundsgatan 9, Gothenburg
Suggestion - weather allowing - to go to an amusement park Liseberg afterwards for Christmas market and enjoyable atmosphere. They feature wonderful skating shows, among others. Beer etc. can also be consumed there, in restaurant Tyrolen


  • Elena Volodina, University of Gothenburg
  • Iztok Kosem, University of Ljubljana
  • David Alfter, University of Gothenburg
  • Jaka Čibej, University of Ljubljana

Contact information

If you have any questions, please contact

  • elena . volodina @ svenska . gu . se or
  • iztok . kosem @ trojina . si


This work has been supported by:

  • enetCOLLECT grant (COST) by covering travel and upkeep of the participants
  • a project grant from the Swedish Foundation for Humanities and Social Sciences (Riksbankens Jubileumsfond: Development of lexical and grammatical competences in immigrant Swedish, project P17-0716:1) through work of the WG leader Elena Volodina and WG member Therese Lindström Tiedemann