List available tokenizers.
getTokenizers()
A character vector with available tokenizers.
Ingo Feinerer
MC_tokenizer and scan_tokenizer.
MC_tokenizer
scan_tokenizer