text_normalizer
text.normalization.text_normalizer(text)Default text normalization function.
Applies - Unicode normalization (NFKC) - Lowercasing - Normalization of whitespace - Remove parentheses and special characters
Parameters
| Name | Type | Description | Default |
|---|---|---|---|
| text | str | Input text to normalize. | required |
Returns
| Name | Type | Description |
|---|---|---|
| tuple | Tuple containing (normalized_tokens, mapping). |