Caching can turn repeated expensive work into a dictionary lookup, but it can also return stale data or grow memory without bound. Python’s functools module provides two convenient memoization decorators: lru_cache and cache.
functools.cache has been available since Python 3.9. It is effectively an unbounded memoization cache. lru_cache adds a configurable size limit and eviction behavior.
Cache functions, not arbitrary side effects
Memoization works best when a function behaves like a pure function: the result depends only on its arguments.
A good candidate is deterministic parsing:
from functools import lru_cache
@lru_cache(maxsize=512)
def normalize_rule(rule: str) -> tuple[str, ...]:
return tuple(part.strip().lower() for part in rule.split(","))The same input always produces the same output.
A poor candidate is a function whose result depends on hidden external state:
@lru_cache(maxsize=128)
def current_balance(user_id: int) -> int:
return query_database(user_id)The cached balance can outlive the database value.
Caching remote or database reads requires an invalidation or expiration policy. lru_cache does not provide TTL expiration.
Choose between cache and lru_cache
Use cache when the input space is naturally small and bounded:
from functools import cache
@cache
def unit_multiplier(unit: str) -> int:
return {
"s": 1,
"m": 60,
"h": 3600,
}[unit]Use lru_cache when arbitrary inputs could accumulate:
from functools import lru_cache
@lru_cache(maxsize=1024)
def compile_template(name: str) -> str:
return expensive_compile(name)Once the cache reaches maxsize, older entries are evicted according to least-recently-used behavior.
A finite limit is a safer default for services that accept user-controlled inputs.
Arguments must be hashable
The cache key is derived from function arguments, so arguments must be hashable:
@lru_cache(maxsize=128)
def summarize(values: tuple[int, ...]) -> int:
return sum(values)Mutable lists cannot be used directly as cached arguments. Convert mutable collections to an immutable representation before calling the cached function when that conversion preserves the operation’s semantics.
Do not normalize blindly if ordering or duplicate values carry meaning.
Normalize equivalent call shapes
Applications sometimes have several call forms that should share one entry. Do not rely on memoization to canonicalize every spelling automatically.
For example, normalize identifiers before the cached boundary:
def get_schema(name: str) -> object:
return _get_schema(name.strip().lower())
@lru_cache(maxsize=256)
def _get_schema(normalized_name: str) -> object:
return load_schema(normalized_name)That keeps cache keys predictable and reduces duplicate entries.
Methods include self in the key
Decorating an instance method caches using the instance as part of the key:
class Parser:
@lru_cache(maxsize=128)
def parse(self, text: str) -> object:
...The cache is attached to the function descriptor and can retain references to instances through cache keys.
That may be fine for long-lived objects, but it can keep otherwise disposable instances alive longer than expected.
For per-instance caching, consider an explicit instance-owned dictionary or cached_property when the computation has no parameters.
Inspect cache behavior
lru_cache and cache expose cache_info():
info = normalize_rule.cache_info()
print(info.hits, info.misses, info.maxsize, info.currsize)This is useful during performance testing.
A cache with almost no hits adds complexity without reducing work. A cache that stays at its maximum size under untrusted input may be thrashing.
Do not log full cache keys when arguments can contain sensitive data.
Clear the cache explicitly
Both decorators expose cache_clear():
normalize_rule.cache_clear()This is useful in tests and for application-level invalidation after configuration reloads.
Be deliberate about when clearing is safe. A global clear can create a burst of expensive recomputation if many requests arrive simultaneously.
Concurrency semantics
The cache data structure is protected so concurrent updates remain coherent. However, more than one thread can call the underlying function for the same key if concurrent misses race before a result is stored.
Do not assume the decorator provides single-flight behavior.
If duplicate execution is unsafe or extremely expensive, add a separate coordination mechanism.
Exceptions are not successful cached results
If the wrapped function raises, that exception is not stored as a successful value. A later call executes the function again.
This is often helpful for transient failures, but repeated invalid input can repeatedly trigger expensive validation unless invalid cases are handled separately.
Avoid mutable cached return values
Every cache hit returns the stored object reference. If callers mutate that object, later callers observe the mutation.
Prefer immutable return types or copy the value before mutation when shared state would be surprising.
Common mistakes
Using cache for unbounded user input
An unbounded cache can grow for the lifetime of the process.
Caching time-dependent functions
Functions that depend on current time, environment variables, files, or database state can become stale unless that state is represented in the key or explicitly invalidated.
Treating hit rate as the only metric
Also measure memory use, miss cost, latency distribution, and invalidation behavior.
Using memoization as a distributed cache
lru_cache is process-local. Multiple workers maintain separate entries, and restarting a process clears them.
A safer memoization checklist
Before adding a decorator, ask:
- Does the result depend only on explicit arguments?
- Are arguments hashable and normalized?
- Is the key space bounded?
- Is stale data acceptable?
- Can returned objects be mutated?
- Is duplicate execution on concurrent misses safe?
- How will tests reset the cache?
- Can
cache_info()show whether the optimization helps?
Memoization is most effective when the function boundary already has clear semantics. The decorator then becomes a small optimization instead of hidden application state.