09 - The Netpoller

🎁 You wrote a TCP server that starts a goroutine per connection and blocks on conn.Read in each one. A blocking read should freeze an entire OS thread, so ten thousand connections should need ten thousand threads. Go runs them on a handful. How?

This is the lesson that explains why Go is so good at network services. The answer is a piece of the runtime called the netpoller, working together with the scheduler. Understanding it turns "goroutine per connection" from a leap of faith into an obvious choice.

The Blocking Thread Problem

An OS thread is expensive: it carries a large stack and the kernel schedules it. If every blocked read held a thread hostage, a server with 10,000 idle-but-open connections would need 10,000 threads, each mostly asleep. That's the classic "C10k problem," and it's why older servers used awkward callback-based event loops. The naive one-thread-per-connection model simply doesn't scale.

Go's promise is that you get to write the simple blocking code and still get the scalability of an event loop. That requires some machinery underneath.

Goroutines, Threads, and Processors

The Go scheduler juggles three things, often called the GMP model:

• G, a goroutine: your unit of concurrent work, cheap to create
• M, a machine: an actual OS thread that runs code
• P, a processor: a scheduling context, with GOMAXPROCS of them

The scheduler multiplexes many Gs onto a small number of Ms. A P is the permission slip an M needs to run Go code, so at most GOMAXPROCS goroutines run truly in parallel. If you want the mental refresher on goroutines themselves, see Concurrency Refresher. The key question is what happens when a running G blocks on I/O.

Parking on a Blocked Read

Here's the trick. Under the hood, Go sets network sockets to non-blocking mode. When your goroutine calls conn.Read and no data is ready, the kernel would normally return EAGAIN (the "try again later" errno from The OS Interface). Instead of spinning, the runtime parks the goroutine and registers its file descriptor with the netpoller (which is epoll on Linux, kqueue on the BSDs and macOS). The M is now free to pick up a different runnable goroutine.

When data eventually arrives on that fd, the netpoller notices and marks the goroutine runnable again. It resumes right where it left off, inside conn.Read, as if the read had simply blocked.

func handleConn(conn net.Conn) {
    buf := make([]byte, 1024)
    n, _ := conn.Read(buf)   // looks blocking; actually parks the goroutine
    conn.Write(buf[:n])
}

You wrote a plain, blocking-looking read. The runtime quietly turned it into an event-driven wait, and no OS thread sat idle during it.

Why Goroutine per Connection Scales

Now the TCP lesson's advice makes sense. Ten thousand connections means ten thousand goroutines, but only the handful that actually have data ready are running on threads at any instant. The rest are parked in the netpoller, costing little more than their small stacks. You get event-loop efficiency with straight-line code:

for {
    conn, _ := ln.Accept()
    go handleConn(conn)   // 10k of these is fine
}

No callbacks, no state machines, no manual epoll bookkeeping. That's the whole payoff.

File I/O Is Different

The netpoller handles sockets, pipes, and other pollable descriptors. Regular files are not pollable on Linux: a read from a file on disk can't be registered with epoll. So when a goroutine makes a genuinely blocking file syscall (or a cgo call), the runtime handles it differently: that M blocks in the kernel, and the scheduler hands the P to another M so other goroutines keep running. If needed, the runtime spins up additional threads.

This is why heavy disk I/O behaves differently from network I/O, and why a program doing lots of blocking file reads may use more OS threads than a purely network-bound one. It still works, but the netpoller isn't the thing saving you there.

Pinning a Goroutine with LockOSThread

Occasionally you need a goroutine to stay on one specific OS thread, because some state lives in the thread itself: certain syscalls, Linux namespaces (the isolation feature containers are built on, entered with setns), thread-local state in a C library, or an OpenGL context. runtime.LockOSThread pins the calling goroutine to its M until you unlock:

runtime.LockOSThread()
defer runtime.UnlockOSThread()
// this goroutine now owns its OS thread for thread-local OS state

Use it sparingly. A locked goroutine ties up a whole thread, which is exactly the cost the scheduler normally works to avoid, so reach for it only when thread-local OS state genuinely requires it.

Observing the Runtime

A few functions let you see the machinery at work. runtime.NumGoroutine counts live goroutines, runtime.NumCPU reports available cores, and runtime.GOMAXPROCS(0) returns the current parallelism limit without changing it:

fmt.Println(runtime.NumCPU())        // 8    cores available
fmt.Println(runtime.GOMAXPROCS(0))   // 8    goroutines that can run in parallel
fmt.Println(runtime.NumGoroutine())  // 3    goroutines alive right now

Watching NumGoroutine climb as connections arrive, while thread count stays flat, is the netpoller doing its job. We'll dig further into inspecting a running process in the next lesson.

Key Takeaways

• OS threads are expensive; one-thread-per-connection doesn't scale (the C10k problem). Go lets you write blocking code and still scale.
• The scheduler multiplexes many Goroutines onto few M (OS threads) via P processors (GOMAXPROCS of them); at most GOMAXPROCS goroutines run in parallel.
• Network sockets are set non-blocking; a would-be-blocking read parks the goroutine and registers its fd with the netpoller (epoll/kqueue), freeing the M. The netpoller wakes the goroutine when the fd is ready.
• This is why goroutine-per-connection scales: parked goroutines cost little, only ready ones use threads.
• Regular file I/O isn't pollable; blocking file/cgo calls block an M and the scheduler moves the P to another thread instead of using the netpoller.
• runtime.LockOSThread/UnlockOSThread pin a goroutine to its thread for thread-local OS state; use sparingly.
• Observe with runtime.NumGoroutine, runtime.NumCPU, runtime.GOMAXPROCS(0).

🎁 Your program can now handle thousands of connections, but how do you know how many file descriptors it's holding, how much memory the OS thinks it's using, or how close it is to a system limit? The answers are sitting in a set of virtual files the kernel keeps just for you.