Linux io_uring Separates Submission From Completion Ownership

An io_uring request can remain in flight after the application has finished constructing its submission queue entry. That creates a lifetime boundary absent from a simple synchronous call: request metadata may become stable at submission, while memory used as the actual I/O payload can still be accessed until the operation completes. The completion queue is therefore not only a result channel. For many operations, it marks the point at which application-owned operation state can be reclaimed or reused.

The distinction matters because the submission queue and completion queue are shared ring structures, but they represent different phases. An SQE describes work to issue. A CQE reports the result of work that reached a completion state. Treating those phases as one ownership interval can produce buffer reuse races, stale correlation data, or queue pressure that is invisible at the call site that prepared the request.

Submission transfers a description, not every referenced byte

An SQE contains fields that describe an operation: opcode, file descriptor, offset, length, flags, and values such as user_data. Some operations also point at application memory.

On kernels that provide stable submission semantics, auxiliary structures used only to describe a request need to remain valid through successful submission rather than through final completion. The data being transferred has a different lifetime. A buffer supplied to an in-flight read or write must remain suitable for the operation until the kernel has finished using it.

That difference is visible in a write request:

struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_write(sqe, fd, buf, len, offset);
io_uring_sqe_set_data(sqe, op);

io_uring_submit(&ring);

Returning from io_uring_submit() does not mean buf is available for arbitrary reuse. It means submission processing has advanced according to the ring mode and library contract. The write result arrives separately as a CQE.

This is an ownership rule, not a property of C pointer syntax. The pointer remains an ordinary userspace address, yet the operation has an outstanding dependency on the referenced storage.

CQEs correlate asynchronous results with application state

Completion order does not have to match the order in which independent requests were submitted. An application therefore needs correlation state that does not depend on queue position.

The SQE user_data field exists for this purpose. Its 64-bit value is copied into the corresponding CQE. Applications commonly place an identifier there or encode a pointer to an operation object whose lifetime extends through completion.

A completion loop can then recover state explicitly:

struct io_uring_cqe *cqe;

if (io_uring_wait_cqe(&ring, &cqe) == 0) {
    struct operation *op = io_uring_cqe_get_data(cqe);

    op->result = cqe->res;
    finish_operation(op);

    io_uring_cqe_seen(&ring, cqe);
}

The queue position carries transport ordering inside the CQ. user_data carries application identity. Conflating those roles creates a fragile assumption that request number N must complete before request number N+1.

Completion consumption is separate from operation completion

A CQE can already exist in the completion ring while userspace has not processed it. At that point the kernel has produced a result, but the application still owns bookkeeping associated with consuming that result and advancing the CQ head.

This creates two related boundaries. Operation completion releases dependencies that were required only while the kernel operation was in flight. CQ consumption releases the completion-ring slot for reuse according to the ring protocol.

Applications that stop consuming CQEs can therefore create backpressure even if the underlying I/O operations themselves finish. Ring sizing changes capacity, not the need to drain completions.

The shared-memory design also means queue head and tail updates require the ordering semantics defined by the interface. Implementations using liburing normally rely on the library helpers rather than open-coding those memory-order details.

Multishot requests extend the completion lifetime

The common one-shot model maps one submitted request to one completion. Multishot operations deliberately change that relation: one SQE can produce multiple CQEs.

For a multishot request, IORING_CQE_F_MORE indicates that more completions may follow from the same request. A CQE without that flag marks the end of that multishot sequence. Application state tied to the persistent request therefore cannot be reclaimed merely because the first CQE arrived.

This changes the useful lifetime model from:

submit -> one completion -> reclaim

to:

submit -> completion + MORE
       -> completion + MORE
       -> final completion
       -> reclaim

The flag is part of the completion contract. Counting the first CQE as final would turn a persistent kernel request into a userspace use-after-free risk if later completions still carry the same correlation value.

Cancellation is also completed asynchronously

Requesting cancellation does not erase an operation from history at the instant the cancellation request is submitted. Cancellation itself is represented as an operation, and races remain possible between target progress and cancellation processing.

The application must interpret the completion results for the target and cancellation request according to the specific opcode contract. A target may already have completed, may be canceled, or may be in a state where the cancellation attempt cannot produce the desired effect.

For resource ownership, the important boundary remains observable completion state rather than the moment cancellation was requested. Reusing a target buffer immediately after issuing cancellation would assume a synchronization guarantee that the cancellation submission alone does not provide.

Shared rings move synchronization into the data structure

Traditional synchronous I/O places request issue, blocking, and return value around one syscall boundary. io_uring can separate those events across queue publication, kernel execution, and later completion consumption. Batching and polling modes can further change when syscalls occur without removing the ownership transitions.

That separation is the central engineering constraint. SQ capacity limits outstanding publication space, CQ capacity limits unconsumed result space, and application memory can have lifetimes that span both user and kernel activity. The interface gains flexibility by making these phases explicit.

A robust design therefore attaches lifetime to operation state rather than to the lexical scope that prepared an SQE. Submission establishes that work has been handed to the ring. Completion reports its result. CQ consumption returns completion capacity. Keeping those boundaries distinct prevents queue mechanics from silently becoming memory-ownership bugs.