Designing APIs for Observable Behavior

An API can keep every documented promise and still break its users.

Suppose a function returns search results with no documented ordering guarantee. Its current implementation happens to return items alphabetically. A client notices that behavior and removes its own sorting step. Months later, the implementation changes and returns the same items in a different order. The API still satisfies its written contract, but the client breaks.

This is the practical problem behind Hyrum’s Law: when an API has enough consumers, some consumers are likely to depend on almost any observable behavior, whether or not that behavior was intended as part of the contract.

The useful lesson is not that every implementation detail must become permanent. It is that API design has two surfaces to reason about: what you explicitly promise and what consumers can repeatedly observe. This article shows how to distinguish them, identify accidental contracts, and make changes with less compatibility risk.

Separate the written contract from observable behavior

An API contract describes the behavior consumers are entitled to rely on. It may specify accepted inputs, returned values, errors, ordering, timing constraints, or other guarantees.

Observable behavior is broader. It includes anything a consumer can detect from outside the implementation.

Consider this simple interface:

findUsers(query) -> list of matching users

Assume the documentation promises only that every returned user matches query. The implementation currently returns users in ascending username order.

The ordering is observable even though it is not promised:

findUsers("dev")

current result:
[devon, devraj, devyn]

A consumer may write:

users = findUsers("dev")
showFirst(users[0])

If the consumer assumes the first item is alphabetically first, the implementation’s incidental ordering has become a dependency for that consumer.

A useful mental model is:

Documentation controls what you intend to promise. Observation controls what consumers are able to depend on.

Those sets often overlap, but they are not automatically identical.

Accidental contracts form through repeated observation

Consumers rarely inspect an implementation and deliberately choose every dependency. Accidental contracts often emerge because a behavior is stable long enough to look intentional.

Imagine an endpoint that returns validation errors:

validateRegistration(input)

The contract says it returns all validation errors. The implementation checks fields in this order:

email
password
age

As a result, errors also appear in that order. A UI test records the exact response:

[
  "email is invalid",
  "password is too short",
  "age is required"
]

The test now encodes ordering even though the application only needs the set of errors. If the validation implementation later checks age first, the UI test fails.

The cause-and-effect chain is important:

1. the implementation exposes a stable observable detail;
2. a consumer finds that detail convenient;
3. the consumer encodes it in code, tests, scripts, or operational assumptions;
4. changing the detail now has a compatibility cost.

The dependency can exist even when the provider never intended to create it.

Not every observable detail deserves the same concern

Treating every visible behavior as permanent would make useful change impossible. Instead, estimate how likely a behavior is to attract dependencies and how expensive those dependencies would be to break.

Three questions help.

Is the behavior easy to observe repeatedly?

Stable response ordering is easier to depend on than an internal allocation strategy. Exact error text is easier to capture than a private helper method.

Behaviors visible through normal API use deserve more attention because consumers do not need unusual techniques to discover them.

Does depending on it simplify consumer code?

Consumers have incentives. If an incidental behavior saves work, some consumer may rely on it.

For example, if results consistently arrive sorted, omitting a client-side sort is tempting. If identifiers are always sequential, code may infer ordering from them even when identifiers were meant to be opaque.

Would changing it produce silent or confusing failure?

A change that causes an immediate validation error is easier to detect than one that subtly changes which record a client chooses.

Compatibility risk is therefore not only about the probability of a dependency. The consequence of breaking that dependency matters too.

Make important guarantees explicit

If consumers genuinely need a behavior, documenting it turns an accidental dependency into a deliberate contract.

Suppose result ordering matters to most callers. Instead of allowing the implementation’s current ordering to act as an unofficial convention, define it:

findUsers(query)

Returns matching users ordered by username ascending.

Now both sides know what must remain stable. Implementations may change freely as long as they preserve that ordering.

Alternatively, expose the choice:

findUsers(query, orderBy)

This makes ordering an explicit capability rather than a side effect of storage or traversal order.

Do not promote every observed detail into the contract. Each guarantee reduces implementation freedom. Promise behavior because consumers need it and you are willing to preserve it, not merely because the current implementation happens to provide it.

Deliberately leave unspecified behavior genuinely unspecified

Documentation that says “order is unspecified” helps, but documentation alone may not prevent dependencies if every production response has used the same order for years.

Where practical, tests should avoid accidentally freezing behavior that the contract does not guarantee.

If ordering is irrelevant, a provider test can compare sets rather than exact sequences:

expected = {alice, bob, carol}
actual = findUsers("a")

assert set(actual) == expected

That test protects the intended guarantee without converting the current ordering into a requirement.

Consumer tests should follow the same principle. Assert the behavior the consumer actually requires. Overly exact snapshot tests, serialized fixtures, and golden files can create accidental contracts when they capture irrelevant details.

This does not mean snapshots or exact comparisons are inherently wrong. They are appropriate when the exact representation is itself part of the required behavior. The key is to know which details the assertion is preserving.

Use variation to reveal hidden assumptions

When a behavior is intentionally unspecified, controlled variation can expose consumers that accidentally depend on it.

Suppose a test implementation of findUsers always returns the same order. A consumer with an ordering dependency may pass unnoticed. A test double that returns valid results in different orders can reveal the assumption:

run 1: [alice, bob, carol]
run 2: [carol, alice, bob]

Both results satisfy an unordered contract. A consumer that fails on the second result depends on a guarantee it was never given.

Variation must be used carefully. Random behavior that makes tests irreproducible can create more debugging cost than insight. Prefer deterministic permutations, parameterized cases, or recorded seeds so a failure can be repeated.

The purpose is not to make software unpredictable. It is to test whether consumers tolerate the range of behavior the contract actually permits.

Evolve behavior as a compatibility change when evidence demands it

Sometimes you discover an accidental contract only when you need to change it.

Imagine an API has returned timestamps with millisecond precision for years, although its documentation promises only a valid timestamp. A new implementation naturally produces microsecond precision. Before changing the output, inspect how the API is used.

Consumers may:

• parse the value with a standards-compliant timestamp parser;
• compare the raw string against stored fixtures;
• allocate a fixed-width database column;
• use the text as part of a cache key.

The first consumer may tolerate the change. The others may not.

At that point, arguing that the old precision was “never documented” does not remove the operational risk. The useful question is whether real consumers depend on it.

Possible responses include preserving the old representation, introducing a new version or field, providing a migration period, or coordinating a change with known consumers. Which option is appropriate depends on the number of consumers, your control over them, and the cost of supporting both behaviors.

Compatibility includes more than data shape

API evolution discussions often focus on fields and function signatures because those contracts are easy to see. Consumers can also depend on behavior such as:

• ordering of otherwise equivalent results;
• exact error categories or text;
• case normalization;
• identifier format;
• retry-visible side effects;
• pagination boundaries;
• default values;
• whether an operation is idempotent in practice;
• relative latency when callers encode aggressive timeouts.

These examples do not all deserve formal guarantees. They illustrate the range of observations that can become dependencies.

Some properties, especially timing, are environment-dependent and cannot usually be promised as exact values. If callers need a latency guarantee, define an explicit service objective or timeout contract with clear conditions rather than allowing current performance to become an informal promise.

Internal APIs can accumulate accidental contracts too

The same reasoning applies inside a codebase.

A module may expose a collection whose documentation says nothing about mutability. Callers discover that it is currently mutable and modify it directly. Replacing it with an immutable value later breaks those callers.

An internal function may throw a particular concrete exception because of its current implementation. Tests catch that exact type even though callers only need to know that the operation failed. Changing the underlying library then breaks tests across the repository.

Internal consumers can often be migrated together, so the cost is lower than changing a public API. But large organizations, shared libraries, plugins, and independently deployed services can make an “internal” interface behave much like a public one.

The relevant question is not whether the API is publicly advertised. It is how many independently changing consumers depend on it.

Avoid two opposite mistakes

The first mistake is assuming undocumented means unused.

A provider changes observable behavior without investigating consumers because the documentation never promised it. This preserves conceptual purity at the expense of real compatibility.

The second mistake is freezing everything forever.

A team treats every historical behavior as sacred because someone might depend on it. The API accumulates quirks and becomes increasingly difficult to improve.

A better approach is evidence-driven:

1. define the guarantees consumers should rely on;
2. avoid tests and examples that accidentally advertise irrelevant details;
3. observe how important consumers actually use the API;
4. treat discovered dependencies according to their migration cost and impact;
5. remove accidental behavior when the compatibility cost is acceptable.

This keeps compatibility concerns grounded in engineering trade-offs rather than fear of any change.

Design examples as part of the contract surface

Documentation examples deserve special care because developers often copy them directly.

Suppose an API says ordering is unspecified, but every example shows alphabetically sorted output. Readers may reasonably infer that sorting is meaningful.

If an example contains an incidental behavior, either vary it or explain the guarantee explicitly. Examples teach consumers what normal usage looks like, so they influence dependencies even when normative text says less.

The same applies to generated SDK examples, command output in tutorials, and sample configuration. Repeated examples can make an implementation detail appear intentional.

Know when the simpler approach is enough

Not every function needs a compatibility strategy.

A private helper with one caller can usually change together with that caller. A small application maintained and deployed as one unit can refactor internal interfaces freely when tests cover the change. Adding versions, migration layers, or extensive compatibility machinery in those cases may add cost without reducing meaningful risk.

The need for deliberate compatibility management grows when consumers are numerous, unknown, independently deployed, slow to update, or outside your control.

Use the principle proportionally. The goal is not to prevent change. It is to understand where observable behavior can turn a local implementation change into somebody else’s breaking change.

Conclusion

An API has both an intended contract and a wider observable surface. Consumers can build dependencies on either.

Design important guarantees explicitly. Keep tests from freezing behavior that does not matter. Use controlled variation to expose assumptions where useful. Before changing long-standing observable behavior, investigate whether real consumers rely on it and choose a migration strategy proportional to the risk.

The practical takeaway is simple: when evolving an API, ask not only “Did we promise this?” but also “Could consumers have reasonably learned to depend on this?” That second question reveals compatibility work that documentation alone cannot show.