Build Portable Readiness Loops in Python with selectors

A network service can handle one connection with straightforward blocking calls: accept a client, read a request, write a response, and repeat. The model becomes awkward when one thread must manage many connections at once.

The problem is not that sockets are slow. The problem is that a blocking operation can stop the thread while one connection waits, even though other connections are ready for useful work.

Python’s selectors module provides a higher-level way to wait for I/O readiness across multiple file objects. Instead of asking one socket to block until something happens, you register many sockets and ask the selector which ones are currently ready.

The key mental model is: a selector reports readiness, not completion. A readable socket may still yield only part of an application message. A writable socket may accept only part of an output buffer. Correct event loops therefore combine readiness notifications with nonblocking I/O and explicit per-connection state.

Start with the smallest useful readiness check

A selector tracks file objects and the events you care about. For sockets, the two common events are:

• selectors.EVENT_READ: an operation such as recv() can make progress without waiting for new input.
• selectors.EVENT_WRITE: an operation such as send() can make progress without waiting for more output capacity.

A small example can use socket.socketpair() to create two connected sockets in one process:

import selectors
import socket

left, right = socket.socketpair()
left.setblocking(False)
right.setblocking(False)

with selectors.DefaultSelector() as selector:
    selector.register(left, selectors.EVENT_READ)

    right.sendall(b"hello")

    for key, mask in selector.select(timeout=1):
        if mask & selectors.EVENT_READ:
            data = key.fileobj.recv(1024)
            print(data)

left.close()
right.close()

The selector does not read the bytes. It only tells the program that left is ready for a read operation. The program still calls recv() and decides what the returned bytes mean.

DefaultSelector chooses the best selector implementation Python exposes for the current platform. That lets application code use one interface without hard-coding Linux epoll, BSD/macOS kqueue, or another lower-level mechanism.

Readiness and nonblocking mode belong together

A readiness loop normally uses nonblocking sockets:

sock.setblocking(False)

Without nonblocking mode, a handler can accidentally block the entire event loop after the readiness check.

This can happen because readiness is only a snapshot of the conditions observed by the operating system. By the time your code performs another operation, conditions may have changed. A handler may also try to read repeatedly after consuming all currently available data.

With a nonblocking socket, an operation that cannot proceed immediately raises BlockingIOError instead of putting the thread to sleep.

That gives the event loop a safe rule:

1. wait until the selector reports readiness;
2. perform the corresponding nonblocking operation;
3. stop when the operation would block;
4. return to the selector.

For a single recv() after a read-ready notification, many programs will receive data immediately. Robust code still treats BlockingIOError as a normal boundary condition rather than as an impossible state.

Store connection state in the selector

Real connections need more than a socket reference. A server may need an input buffer, an output buffer, parsing state, or an identifier for logging.

register() accepts an arbitrary data object:

from dataclasses import dataclass, field

@dataclass
class Connection:
    incoming: bytearray = field(default_factory=bytearray)
    outgoing: bytearray = field(default_factory=bytearray)

Register the state together with the socket:

state = Connection()
selector.register(
    client,
    selectors.EVENT_READ,
    data=state,
)

When select() reports the socket, the associated object is available through key.data:

for key, mask in selector.select():
    state = key.data

This keeps per-connection state attached to the registration rather than spread across parallel dictionaries indexed by file descriptor.

Accept connections without blocking the loop

A listening socket can be registered for read readiness. For a listening socket, read readiness means that an accept() can make progress.

import selectors
import socket

selector = selectors.DefaultSelector()

listener = socket.socket()
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind(("127.0.0.1", 8000))
listener.listen()
listener.setblocking(False)

selector.register(listener, selectors.EVENT_READ, data=None)

One useful convention is to reserve data=None for the listener and use a connection-state object for clients:

def accept_ready(listener):
    client, address = listener.accept()
    client.setblocking(False)

    state = Connection()
    selector.register(
        client,
        selectors.EVENT_READ,
        data=state,
    )

The accepted socket is a new file object. It must be made nonblocking and registered separately.

A busy listener can have more than one queued connection. A production server may call accept() repeatedly until it raises BlockingIOError, so one readiness notification can drain all currently pending accepts. The simpler single-accept form is often enough to understand the control flow before adding that loop.

Treat received bytes as a stream, not a message

A common networking mistake is to assume that one recv() call returns one complete application message.

TCP does not preserve application write boundaries. If a peer sends two logical messages, the receiver might observe them separately, combined, or split across several reads.

A read handler should therefore append bytes to a connection buffer:

def read_ready(sock, state):
    try:
        chunk = sock.recv(4096)
    except BlockingIOError:
        return

    if not chunk:
        close_connection(sock)
        return

    state.incoming.extend(chunk)

An empty bytes object has a special meaning for a TCP stream: the peer performed an orderly shutdown of its sending side and there is no more data to read. It is not the same as “nothing is available yet”; nonblocking “not available yet” is represented by BlockingIOError.

After buffering input, a protocol parser can extract complete frames, lines, or length-prefixed messages while leaving incomplete data in the buffer for the next readiness notification.

This separates two concerns cleanly:

• the selector and socket layer moves bytes when I/O can make progress;
• the protocol layer decides when those bytes form complete messages.

Partial writes require an output buffer

Write readiness is also easy to misinterpret.

A writable socket is not a promise that send() will accept the entire response. It means at least some progress can be made under the current conditions. send() returns the number of bytes accepted, and that number may be smaller than the buffer supplied.

For nonblocking event-loop code, keep unsent data in an output buffer:

def write_ready(sock, state):
    if not state.outgoing:
        return

    try:
        sent = sock.send(state.outgoing)
    except BlockingIOError:
        return

    del state.outgoing[:sent]

When the buffer becomes empty, stop watching for write readiness.

That detail matters because sockets are frequently writable. If every idle connection remains registered for EVENT_WRITE, the selector can wake repeatedly even when the application has nothing to send.

Modify the registration only when output is queued:

def update_interest(sock, state):
    events = selectors.EVENT_READ

    if state.outgoing:
        events |= selectors.EVENT_WRITE

    selector.modify(sock, events, data=state)

This is an important event-loop invariant: register write interest because the application has pending output, not merely because the socket supports writing.

Put the pieces into a small connection handler

The read and write paths can share one dispatch function:

def service_connection(key, mask):
    sock = key.fileobj
    state = key.data

    if mask & selectors.EVENT_READ:
        read_ready(sock, state)

    if mask & selectors.EVENT_WRITE:
        write_ready(sock, state)

    if sock.fileno() != -1:
        update_interest(sock, state)

A selector event mask can contain both read and write bits. Use independent if statements rather than if/elif so the handler can process both conditions when both are reported.

The fileno() check above reflects one possible ownership convention: read_ready() may close the socket after EOF, and a closed Python socket reports -1. Another valid design is for handlers to return an explicit “closed” result. The important part is to choose one lifecycle convention and apply it consistently.

A close helper should unregister before closing:

def close_connection(sock):
    try:
        selector.unregister(sock)
    except KeyError:
        pass
    sock.close()

Unregistering removes the file object from the selector’s tracking set. Closing without unregistering first makes lifecycle reasoning harder, especially if the operating system later reuses the same numeric file descriptor for an unrelated socket.

Keep work per event bounded

A readiness loop is cooperative: one thread handles each ready file object in turn.

That means a handler that performs expensive CPU work, sleeps, or loops for a long time delays every other connection. The selector does not make CPU-heavy handlers concurrent.

Keep readiness handlers focused on bounded work:

wait for readiness
    -> move some bytes
    -> update connection state
    -> update event interest
    -> return to the selector

If parsing or computation can be expensive, consider handing that work to another execution mechanism while the event-loop thread remains responsible for socket state. That introduces synchronization and backpressure concerns, so it is not automatically simpler; use it when the workload actually requires parallel or background computation.

Use timeouts for periodic event-loop work

select() accepts a timeout:

events = selector.select(timeout=0.5)

A timeout does not mean a connection timed out. It means the selector waits no longer than that interval for readiness before returning control to your loop.

That can be useful when the same loop must periodically inspect application-level deadlines:

while running:
    for key, mask in selector.select(timeout=0.5):
        if key.data is None:
            accept_ready(key.fileobj)
        else:
            service_connection(key, mask)

    expire_idle_connections()

For many independent timers or strict scheduling requirements, repeatedly scanning all connections may become inefficient or awkward. A priority queue of deadlines or an event-loop framework with timer support can be a better fit.

Know what selectors guarantees—and what it does not

The selectors abstraction gives you a portable interface for readiness notification, but portability has boundaries.

DefaultSelector chooses an implementation available on the current platform; your code should not assume which concrete class was selected.

The kinds of file objects that can be monitored also vary by platform. Python documents that Windows selectors support sockets but not pipes, while Unix selectors support sockets and pipes and may support additional descriptor types. If your program relies on monitoring something other than sockets, verify the target platform rather than assuming socket-level portability extends to every file object.

A selector also does not define your application protocol, buffering limits, fairness policy, connection timeouts, or overload behavior. Those remain application responsibilities.

Avoid the most common readiness-loop mistakes

Leaving every socket registered for writes

A connection that has no pending output usually does not need EVENT_WRITE. Continuous write interest can cause unnecessary wakeups and wasted CPU.

Register write readiness only while state.outgoing contains bytes.

Assuming readiness means a full message is available

Read readiness only means a read can make progress. TCP can split or combine application messages.

Buffer bytes and parse according to an explicit framing rule.

Assuming one send writes the whole buffer

send() reports how many bytes it accepted. Preserve the remainder and wait for the next write-ready event.

For simple blocking code, sendall() can handle this loop for you. In a nonblocking readiness loop, explicitly tracking the remainder gives the event loop control instead of monopolizing the thread until all data is sent.

Doing blocking work inside a handler

A single blocking database call, filesystem operation, DNS lookup, or sleep can stall all connections managed by that event-loop thread.

A selector helps only with the file objects it is actually waiting on. It does not automatically make unrelated blocking operations nonblocking.

Forgetting cleanup paths

EOF, protocol errors, application shutdown, and unexpected exceptions all need a clear path that unregisters and closes the connection.

Resource ownership should be explicit enough that a socket cannot remain registered after the application considers it dead.

When selectors are a good fit

Use selectors when you want a small, synchronous event loop around multiple sockets or supported file descriptors and you are comfortable managing connection state explicitly.

It is especially useful for learning readiness-based networking because the control flow remains visible: register interest, wait, perform nonblocking I/O, update state, and repeat.

A higher-level asynchronous framework is often a better choice when you need DNS integration, subprocess management, timers, cancellation, structured task composition, TLS helpers, or a large ecosystem of asynchronous libraries. Python’s asyncio builds a much broader programming model around event-driven I/O.

Blocking sockets with one thread per connection can also be the simpler solution for modest connection counts or applications dominated by blocking libraries. Readiness-based design has real bookkeeping costs, and avoiding a thread per connection is only valuable when that trade-off matches the workload.

Conclusion

Python’s selectors module is easiest to use correctly when you think in terms of readiness rather than completed operations.

A selector tells you which registered file objects can make progress. Nonblocking socket calls perform that progress without stalling the loop. Per-connection buffers preserve state across partial reads and writes, and dynamic event registration prevents idle sockets from generating needless work.

Those ideas are reusable beyond any particular server: separate transport readiness from protocol completeness, make partial progress explicit, and keep each event-loop turn bounded.