Just some brief notes on .NET async architecture for my own reference.

Async operations are fundamental to the hardware

The ability to do async effectively starts at the hardware level. Functionally, there is no need for blocking I/O on modern hardware where DMA reads from interfaces (network, disk, etc) into memory are possible. The processor does not spin wait, but instead receives an interrupt which signals data requested is ready for read. This means the hardware itself yields time back to the CPU.

See Stephen Cleary’s excellent article There is no Thread

At the OS level

At the OS level, Linux provides for this with two features:

  • Epoll
  • io_uring

IO_Uring is the new one, but .NET (as of 2024) still uses epoll. These send signals from the operating system to the application when data that has been loaded asynchronously from the network (via DMA) is now available for use.

Brief Aside: Windows and IOCP

Windows NT 4 introduced I/O Continuation Ports which are Microsoft’s version of the epoll functionality. This far predated any UNIX releases gaining this functionality, and the Linux kernel itself didn’t introduce it until around 2002. For this period of time in the 1990s, it was acknowledged that Microsoft had the superior implementation for handling large numbers of concurrent connections.

When you block

When you use blocking I/O via an underlying API that does not use modern OS constructs available for async IO (be that io_uring or epoll or something else), you are forcing a thread to wait for the resolution of the resource. Briefly, this means that you have a thread that could be unutilized being utilized for an unnecessary wait. In aggregate, this means you’re running the risk of thread exhaustion.