I/O Completion Ports - Windows API Reference

Introduction

I/O completion ports (IOCP) are a scalable, high-performance mechanism for asynchronous I/O operations in Windows. They allow a multithreaded application to efficiently handle I/O requests by assigning them to a limited number of threads that are responsible for processing completed I/O operations. This approach minimizes thread creation and context switching overhead, making applications more responsive and scalable, especially under heavy I/O loads.

IOCP is particularly useful for server applications that need to handle a large number of concurrent client connections, such as web servers, database servers, and network services. By using IOCP, these applications can achieve high throughput and low latency.

Key Concepts

Completion Port: A kernel object that acts as a queue for completed I/O operations.
I/O Request Packet (IRP): A data structure used by the operating system to manage I/O operations. When an asynchronous I/O operation completes, an IRP is placed on the completion port's queue.
I/O Completion Thread: A thread that waits for I/O operations to complete by calling GetQueuedCompletionStatus. When an IRP is available, the thread processes it.
Worker Thread: A thread that initiates I/O operations and associates them with the completion port.
Association: An I/O handle (e.g., a file handle, socket handle) is associated with a completion port using CreateIoCompletionPort.

Core Functions

The following functions are central to working with I/O completion ports:

CreateIoCompletionPort: Creates a new I/O completion port or associates an existing handle with a completion port.

HANDLE CreateIoCompletionPort(
  HANDLE FileHandle,
  HANDLE ExistingCompletionPort,
  ULONG_PTR CompletionKey,
  DWORD NumberOfConcurrentThreads
);

GetQueuedCompletionStatus: Retrieves I/O status information from a completion port. This function blocks until an I/O operation completes or an error occurs.

BOOL GetQueuedCompletionStatus(
  HANDLE CompletionPort,
  LPDWORD lpNumberOfBytesTransferred,
  ULONG_PTR *lpCompletionKey,
  LPOVERLAPPED *lpOverlapped,
  DWORD dwMilliseconds
);

PostQueuedCompletionStatus: Posts an I/O status information to a completion port. This is useful for signaling threads or sending custom completion notifications.
```
BOOL PostQueuedCompletionStatus(
  HANDLE CompletionPort,
  DWORD dwNumberOfBytesTransferred,
  ULONG_PTR dwCompletionKey,
  LPOVERLAPPED lpOverlapped
);
```
SetFileCompletionNotificationModes: Controls whether a file handle can trigger completion notifications for specific I/O operations.
```
BOOL SetFileCompletionNotificationModes(
  HANDLE FileHandle,
  UCHAR Flags
);
```

Usage Scenario: A Simple Server

A typical server using IOCP would follow these steps:

Create an I/O completion port using CreateIoCompletionPort with NULL for the handle and the desired number of threads.
For each new client connection (e.g., a socket):
- Associate the client's handle with the completion port using CreateIoCompletionPort.
- Post an initial read request for the client using ReadFile or WSARecv with an overlapped structure.
Create a pool of worker threads. Each worker thread will:
- Call GetQueuedCompletionStatus to wait for completed I/O operations.
- When an operation completes, examine the OVERLAPPED structure and handle the data (e.g., process a received message, send a response).
- Post new I/O requests (e.g., read for the next message, write a response) using the associated handle.
Implement a mechanism to gracefully shut down the server, often by posting a special message to the completion port or signaling the worker threads.

Performance Considerations

Note: The optimal number of concurrent threads for an IOCP is often related to the number of CPU cores available on the system. A common recommendation is to set it to twice the number of cores, but this can vary based on the workload.

Overlapped I/O: IOCP is designed for asynchronous, overlapped I/O operations. Synchronous operations will block the worker thread.
Thread Pool Size: Carefully tune the number of worker threads to balance resource utilization and responsiveness.
I/O Size: Batching small I/O requests or using larger, more efficient I/O operations can improve throughput.
Completion Key: Use the CompletionKey parameter in CreateIoCompletionPort and PostQueuedCompletionStatus to associate application-specific data with I/O operations, simplifying message handling.

Introduction

Key Concepts

Core Functions

Usage Scenario: A Simple Server

Performance Considerations

See Also