tokenizer.load_tokenizer
text.tokenizer.load_tokenizer(language='swedish')Loads a PunktTokenizer for the specified language that can be used to sentence tokenize text.
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| language | str | Language to use for the tokenizer, e.g. “swedish”, “english”. | "swedish" |
Returns
| Name | Type | Description |
|---|---|---|
| PunktTokenizer | Loaded tokenizer. |