One sentence, two counts
Take the sentence 'do not split blindly'.
Count spaces plus one and label it a toy.
If you have a real tokenizer, record its name and its count beside the toy.
Sources and go deeper
- Vaswani and co-authors, Attention Is All You Need, 2017 paper last checked 2026-10
- Brown and co-authors, Language Models are Few-Shot Learners, 2020 paper last checked 2026-10
- Attention Is All You Need — Wikipedia wikipedia last checked 2026-10
- Large language model — Wikipedia wikipedia last checked 2026-10