sbx-eng-tokenization-stanza

This analysis is used with Sparv. Check out Sparv's quick start guide to get started!

To use this analysis, add the following line under export.annotations in the Sparv corpus configuration file:

- stanza.token  # Token segments

In order to use this annotation you need to add the following settings to your Sparv corpus configuration file:

metadata:
  language: eng

classes:
  token: stanza.token

For more info on how to use Sparv, check out the Sparv documentation.

Example output:

<token>This</token>
<token>is</token>
<token>a</token>
<token>corpus</token>
<token>.</token>