Transformers

98 articles

Artificial Intelligence 03 Sep 2026 7 min read

Tokenization in Large Language Models

Large language models do not read text as words or characters in the way people do. Before text reaches the model, a tokenizer converts it into a sequence of discrete units called tokens and maps those tokens to numerical identifiers. Tokenization is easy to overlook because most model APIs perform it automatically. Yet token boundaries affect context-window usage, inference cost, truncation, multilingual behavior, and even whether two visually similar strings are represented in similar ways. These details explain many LLM behaviors that otherwise look inconsistent.

Artificial Intelligence LLM

Artificial Intelligence 03 Sep 2026 7 min read

KV Caching in LLM Inference

Large language models generate text one token at a time. Without an optimization, every new token would force the model to repeat attention calculations for tokens it has already processed. KV caching avoids much of that repeated work. During inference, the model stores the key and value representations produced by attention layers for previous tokens. When generating the next token, it can reuse those stored representations instead of recomputing them from the beginning.

Artificial Intelligence LLM