Segmenterar text i meningar utifrån skiljetecken med hjälp av NLTKs RegexpTokenizer
En mycket enkel meningssegmenterare som separerar meningar vid varje .!? oavsett sammanhang. Meningssegmenteraren är baserad på NLTK RegexpTokenizer.
Citering
Språkbanken Text (2021). sentence-punkt (uppdaterad: 2021-09-21). [Analysis]. Språkbanken Text.
Standardreferens
Bird, Steven, Edward Loper and Ewan Klein (2009), Natural Language Processing with Python. O’Reilly Media Inc.
Exempel
This analysis is used with Sparv. Check out Sparv's quick start guide to get started!
To use this analysis, add the following line under export.annotations in the Sparv corpus configuration file:
- segment.sentence # Sentence segments
In order to use this sentence segmenter you need to add the following setting to your Sparv corpus configuration file:
segment:
sentence_segmenter: punctuation
For more info on how to use Sparv, check out the Sparv documentation.
Example output:
<sentence>
<token>Det</token>
<token>här</token>
<token>är</token>
<token>en</token>
<token>korpus</token>
<token>.</token>
</sentence>
<sentence>
<token>Den</token>
<token>har</token>
<token>flera</token>
<token>meningar</token>
<token>.</token>
</sentence>