Concurrency and parallelism get used interchangeably all the time, mostly because both describe a system doing more than one thing. But they solve genuinely different problems, and most of the machinery built around them, threads, event loops, thread pools, virtual threads, exists precisely because neither one is enough on its own.
Through this post we build the whole model up from first principles. Starting with the distinction itself.
Concurrency vs Parallelism #
Concurrency is a system making progress on multiple tasks over the same period of time, without those tasks necessarily running at the same instant. Parallelism is multiple tasks actually running at the same instant, on separate physical cores.
Lets look at a single core juggling two tasks:
Time →
Task A: ███ ███ ███
Task B: ███ ███
At any given instant only one task is executing, and yet both are making progress. This is concurrency: the illusion of simultaneity, produced by rapid switching.
Now compare that to two cores:
Core 1 → Task A
Core 2 → Task B
Here both tasks are genuinely executing at the same instant. This is parallelism, and it requires actual hardware: multiple cores, multiple processors.
You might assume the two always go together. They don't! You can have concurrency without any parallelism at all, a single core can juggle thousands of tasks without ever running two of them at once. Parallelism, on the other hand, needs concurrency's scheduling ideas plus the physical means to run more than one task simultaneously.
So if concurrency doesn't need multiple cores to be useful, what's it actually buying us? That depends entirely on what the tasks are spending their time doing.
The Waiting Problem #
Every program sits somewhere between two extremes.
I/O bound programs spend most of their time waiting: on a database response, a network round trip, a filesystem read, a disk seek. The CPU is idle for most of this. API servers, chat backends, most of what a web backend does day to day, all fall here. Concurrency is a huge win for this class of work, because while one task sits blocked on I/O, the CPU can be handed straight to another task that actually has something ready to do.
CPU bound programs spend most of their time computing: image processing, encryption, video encoding, compression, running inference. Here the CPU itself is the bottleneck, not some external system, and no amount of clever scheduling changes the fact that the computation has to happen. Parallelism, real simultaneous execution across cores, is the only thing that actually helps.
This gives us a clean rule of thumb: concurrency hides latency, parallelism increases throughput. Apply the wrong one and you either get nothing (parallelizing something that was waiting on the network, not computing) or you fail outright (trying to concurrency your way out of a computation that has to run, full stop).
With that framing sorted, we can get into the actual primitive most systems reach for first: the thread.
Threads, and Why They're Expensive #
A thread is the smallest unit of execution the OS scheduler actually knows about. Each one carries its own stack, its own program counter, its own registers. Threads in the same process share the heap, globals, and file descriptors:
Process
├── Thread 1
├── Thread 2
└── Thread 3
Shared:
Heap
Globals
Files
That shared state is exactly what makes threads useful for concurrent work inside a process, and exactly what makes them dangerous, more on that later. First we need to understand why threads can't just be the default answer to "handle more concurrent work," and that comes down to cost.
- Memory. Every thread needs its own stack, usually around a megabyte. Ten thousand threads at a megabyte each is already something like ten gigabytes committed, before a single one of them has done anything useful.
- Creation cost. Spinning up a thread is a system call: kernel allocation, scheduler registration, stack setup. Not free, and doing this per unit of work at scale means paying fixed overhead before the real task even starts.
- Context switching. Every switch between threads means saving the current thread's registers and stack pointer, then loading the next one's. Past a certain number of runnable threads, the CPU spends more and more of its cycles on this bookkeeping instead of actual work.
So a thread-per-task model clearly breaks down under load. What's the alternative? This is where the event loop comes in.
The Event Loop, from First Principles #
Suppose instead of allocating a thread per concurrent task, we keep exactly one thread and give it a queue of work, plus a way to get notified when I/O finishes. That's the whole idea behind an event loop.
One Thread
↓
Event Loop
↓
Network
Database
Filesystem
Timers
When an async operation kicks off, the thread doesn't sit around waiting on it. It hands the operation to the OS and moves on:
Read file()
↓
OS handles I/O
↓
Thread is free
↓
Callback is queued
↓
Event Loop executes callback
The thread frees up the instant it issues the operation, and only comes back once there's a callback ready to run. Zoom out to request handling and the same pattern holds:
Incoming Request
↓
Start async operation
↓
Return immediately
↓
Continue processing other requests
↓
Operation finishes
↓
Callback enters queue
↓
Event loop executes callback
This is the whole reason a single threaded runtime like Node.js can hold thousands of open connections at once. It's never blocked on any one of them, it's only ever context switching logically between queued callbacks, without paying the real OS level cost of an actual thread switch every time.
Operations That Refuse to Be Async #
Here's the catch: the event loop model assumes the OS can just tell us when I/O finishes, without blocking a thread to find out. Not everything plays along. Filesystem calls, DNS resolution, some crypto and compression routines, these often have no genuine non blocking path at the OS level at all.
Node's answer, through libuv, is a small pool of worker threads sitting behind the event loop specifically to absorb this category of work:
Event Loop
↓
Thread Pool
Worker 1
Worker 2
Worker 3
Worker 4
Default pool size is four threads, and it's configurable. It's a pragmatic patch: keep the main thread free, and quietly hand off the small set of operations that genuinely need a blocking OS thread to a small, bounded pool, instead of spinning up new threads on demand.
Virtual Threads: A Different Answer to the Same Problem #
The event loop is one way to deal with "threads are too expensive to hand out per task." Java's virtual threads get to a similar place through a completely different route.
Normally, one Java thread maps to one real OS thread, inheriting the entire cost profile above. Virtual threads break that mapping apart: thousands of Java threads can sit on top of a handful of OS threads, with the JVM doing the scheduling instead of the kernel.
What that buys you in practice: blocking style code, the kind that's easiest to write and reason about, becomes cheap again, because blocking a virtual thread doesn't block the real thread underneath it. High concurrency, low memory, and you don't have to restructure your code around callbacks to get there.
Shared State: Where Things Get Dangerous #
Everything so far, threads, event loops, thread pools, virtual threads, is about letting multiple units of work make progress over the same window of time. None of it says anything about what happens when that work touches the same memory.
A race condition happens when multiple threads read and write shared memory with no coordination at all:
count = 0
Thread A:
read count
Thread B:
read count
Thread A:
count++
Thread B:
count++
Expected:
2
Actual:
1
Both threads read zero before either one writes back. Each increments its own stale copy, and one write clobbers the other. Expected two, actual one, and the annoying part is this bug is often completely invisible in normal testing, only showing up once real concurrent load makes the bad interleaving likely.
So how do we actually fix this? The answer, every time, is to make a read, modify, write sequence behave as one atomic unit. What differs is the tool.
Synchronization Primitives #
- Mutex. Grants exclusive access to a critical section to exactly one thread at a time. Everyone else blocks until it's released. Simple, but a bad fit under high contention, since threads spend more time waiting than working.
- Semaphore. Same idea as a mutex, but generalized to let up to N threads in at once instead of exactly one. Good for bounding access to a limited resource pool, capping concurrent DB connections for example, rather than enforcing strict single access.
- RWLock. Splits readers from writers: any number of readers can hold the lock together, but a writer needs it exclusively. A real win for read-heavy workloads, where a plain mutex would be serializing reads that never actually conflicted with each other.
- Condition Variable. Pairs with a mutex to let a thread sleep until some condition becomes true, instead of busy-waiting and burning cycles checking over and over. Whoever updates the shared state signals the variable, waking whoever was waiting on it.
- Atomic operations. Some operations, increment, compare-and-swap, are a single indivisible CPU instruction. No lock needed at all, because there's no window where another thread could observe a half finished state.
- Channels / message passing. Instead of sharing memory and fighting over access to it, threads (or goroutines) send values through a channel. This sidesteps the whole problem structurally: if state is never shared, it can't race. Go leans on this hard, and its own tagline says it best: don't communicate by sharing memory, share memory by communicating.
- Immutable data. If shared data is never mutated after creation, there's nothing left to coordinate, since concurrent readers have nothing to race against. You're trading runtime synchronization cost for upfront design cost.
Processes vs Threads #
Everything above lives inside a single process. Worth stepping back one level to see why threads exist as a separate idea from processes at all.
A process gets its own isolated memory space, its own file descriptor table, its own address space, entirely separate from every other process. Threads inside a process share all of that instead. That gives each model a different tradeoff:
- Processes are isolated but expensive to spin up and expensive to talk between, since crossing the process boundary usually means IPC, pipes, sockets, shared memory, rather than a direct reference. A crash in one doesn't directly corrupt another.
- Threads are cheap relative to processes and can just read and write shared memory directly, but that same sharing means a bad write or a race condition in one thread can corrupt state the whole process depends on, and a fatal error in one thread can take the entire process down with it.
So the choice really comes down to isolation. Multi-process setups (worker pools, sandboxed execution) trade some performance and simplicity for fault isolation. Multi-threaded setups trade fault isolation for lower overhead and simpler shared access. Most real systems end up doing both: processes for the isolation boundary, threads or an event loop inside each one for the concurrency.
Async/Await, Under the Hood #
async/await reads about the same across every language that supports it, but it compiles down to real different runtime machinery depending on where you are.
Node.js. An async function is sugar over a promise. await doesn't block the thread, it suspends the function and registers a continuation that the event loop resumes once the promise settles. There's exactly one thread doing all this scheduling, the same event loop from earlier, and every await is a point where control hands back to that loop to process other queued work before coming back to you.
Go. No async/await at all here, Go goes with goroutines and channels instead. A goroutine is a lightweight, runtime managed unit of execution, cheaper than an OS thread, that the Go runtime multiplexes across a small number of real OS threads. Blocking a goroutine on I/O doesn't block the OS thread under it, the scheduler just moves other goroutines onto it. Conceptually this is basically Java's virtual threads: lots of logical units running over a few real ones, runtime doing the scheduling.
Java. Before virtual threads, async in Java usually meant CompletableFuture chains or a reactive library, explicitly composing callbacks instead of writing something that reads sequentially. Virtual threads change that math directly: blocking a virtual thread is cheap now, so plain blocking style code becomes viable again, no callback composition, no reactive style required.
The thread tying all three together (pun intended): the goal is always to avoid tying up a scarce OS thread on something that's mostly waiting. What differs is what's waiting, on what, and whose job it is to resume it, an event loop and callback queue, a runtime scheduler juggling goroutines, or a JVM scheduling virtual threads onto carrier threads underneath.
Advantages and Tradeoffs #
- Concurrency hides latency by interleaving tasks that spend time waiting. It does not need multiple cores to be useful.
- Parallelism increases throughput by actually running tasks at the same instant. It needs multiple cores and is the right tool for CPU bound work specifically.
- Threads are the default concurrency primitive but carry real memory, creation, and context switching cost, which makes a thread-per-task model expensive fast.
- Event loops, thread pools, and virtual threads are all different answers to the same question: how do you get high concurrency without paying the full OS thread cost for every unit of work?
- Shared mutable state is the one thing none of the above solve for free. Mutexes, semaphores, RWLocks, condition variables, atomics, channels, and immutable data are the actual tools for making concurrent access safe, each trading off differently on cost and contention.
- Processes buy isolation at the cost of performance and communication overhead. Threads buy performance at the cost of isolation. Most systems end up using both, at different layers.
last updated: 2026-07-17
published with prose.sh