TL;DR: Three bugs appear in almost every first attempt. Putting one sentinel on the queue to stop all consumers, which stops exactly one, because whichever consumer takes it exits and the others block forever: put one sentinel per consumer. Letting an exception escape the consumer loop, which silently reduces the worker count until throughput drops for no visible reason and eventually nothing drains the queue: wrap the work in try/except inside the loop and record the failure. And treating the bound as a memory limit rather than as backpressure, which leads to a queue of 100,000 that never fills and therefore never slows the producer, so a fast producer and a slow consumer accumulate work until the process dies. The bound should be small, a few times the consumer count, so that a full queue blocks the producer promptly and the pipeline runs at the consumer's rate rather than at the producer's.
How to approach it
Write the correct version and then name the three bugs against it, because they are what the question is about. Show the measurement that proves backpressure is working. Then the shutdown ordering, which is the part that is easy to get subtly wrong even after fixing the sentinel count. Close with the choice of threads against asyncio, since the answer depends on what the work does.
A strong answer
A typical situation: a preprocessing pipeline runs with four consumer threads and an unbounded queue. Throughput is fine for an hour, then memory climbs steadily and the process is killed. The producer reads from fast local storage and the consumers do CPU work, so the producer outruns them by a factor of three and every item it produces beyond that accumulates.
The implementation:
import queue, threading
SENTINEL = object()
def run_pipeline(items, work, n_consumers=4, maxsize=8):
q = queue.Queue(maxsize=maxsize) # small: this is backpressure, not storage
results, errors = [], []
lock = threading.Lock()
def producer():
try:
for item in items:
q.put(item) # blocks when full: the backpressure
finally:
for _ in range(n_consumers): # ONE SENTINEL PER CONSUMER
q.put(SENTINEL)
def consumer():
while True:
item = q.get()
try:
if item is SENTINEL:
return
with lock:
results.append(work(item))
except Exception as exc: # a failing item must not kill the worker
with lock:
errors.append((item, repr(exc)))
finally:
q.task_done()
p = threading.Thread(target=producer)
cs = [threading.Thread(target=consumer) for _ in range(n_consumers)]
p.start()
for c in cs: c.start()
p.join()
for c in cs: c.join()
return results, errors
Running it with 50 items, 4 consumers, a queue of 5, and one item that raises:
processed 49, errors 1 -> [(13, "ValueError('bad item 13')")]
queue never exceeded 5 items, so the producer was held at the consumers' rate
The three bugs:
bug 1: one sentinel for all consumers
wrong: q.put(SENTINEL) once, after the loop
what happens: one consumer takes it and returns; the other three block on q.get() forever;
the join never completes and the program hangs at exit
right: one sentinel per consumer, so each one receives exactly one
the subtlety: this only manifests with more than one consumer, so a test with a single
consumer passes
bug 2: the exception escapes the loop
wrong: the work call is not wrapped, so an exception propagates out of consumer() and the
thread dies
what happens: throughput drops by 1/n with no error visible unless the thread's exception is
being logged, and after n failures nothing drains the queue and the producer blocks forever
right: try/except around the work, inside the while loop, recording the failure
the subtlety: this looks like a slowdown rather than a crash, which is why it survives
bug 3: the bound is too large to be backpressure
wrong: queue.Queue(maxsize=100_000) or queue.Queue() with no bound
what happens: the queue never fills, so put never blocks, so the producer runs at full speed
and the difference between producer and consumer rates accumulates in memory
right: a small bound, a few times the consumer count, so the queue fills quickly and the
producer is held
the arithmetic: with a producer at 3,000 items/s and consumers at 1,000/s, an unbounded
queue grows by 3,000 - 1,000 = 2,000 items per second
at 10 KB per item that is 2,000 x 10 KB = 20 MB/s of growth
against 16 GB of headroom = 16e9 / 20e6 = 800 s, so the process dies after about 13 minutes
a bound of 8 makes put block, so the pipeline runs at the consumers' 1,000/s and memory
is flat
sanity: the failure takes 13 minutes to appear, which is long enough to pass every test and
short enough to happen in production on the first real dataset
Producer-Consumer Pipelines covers the pattern; Batching Queues and Backpressure covers the same idea where the consumer is a remote service.
Shutdown ordering, which stays subtle after the sentinel fix:
the sentinels go in the producer's finally block
why: if the producer raises partway through, the consumers must still be told to stop, or
the program hangs on join with a traceback already printed
join the producer before the consumers
the producer's finally has queued the sentinels by the time it returns, so consumers are
guaranteed to see them
what about task_done and q.join()
useful when the producer needs to know all work is complete before signalling shutdown
the version above does not need it, since the sentinels already order the shutdown, but the
finally block calls task_done unconditionally so the counter stays consistent if a caller
does use q.join()
what must never happen
a consumer returning without calling task_done, which leaves q.join() hanging
a sentinel consumed by a consumer that then continues, which leaves another consumer without
one
Threads or asyncio, which decides the shape:
threads
right when the work releases the interpreter lock: file and network I/O, and numpy or other
extension code that releases it during computation
the queue module is thread-safe and blocking, which is what the code above uses
asyncio
right when the work is I/O awaiting other services, with many concurrent operations
asyncio.Queue with the same structure, and the sentinel and exception rules are identical
processes
right when the work is pure Python CPU, since threads there contend on the interpreter lock
and give no parallelism; the queue becomes a multiprocessing queue and items must be
picklable, which changes what can be passed
Concurrency in Python, Go and C++ compares the three models.
The reversal condition: an unbounded queue is correct when the producer is inherently slower than the consumers and the bound would only add lock contention for no benefit, for example a producer reading from a network at 100 items per second feeding consumers that handle thousands. The test is whether the queue ever reaches its bound in practice: if it never does, the bound is doing nothing and could be anything, and if it frequently does, the bound is the mechanism holding the pipeline together. Measuring the queue's high-water mark answers that in one run and is worth instrumenting for exactly that reason.
What interviewers probe next
- "How do you stop early on the first error?" A shared cancellation flag checked by both sides, plus draining the queue so the producer's put does not block forever on a queue nobody is reading.
- "What if the producer is also several threads?" A counter of live producers; the last one to finish queues the sentinels. Otherwise the first producer to finish shuts down the consumers.
- "How would you measure whether backpressure is working?" The queue's high-water mark. Consistently at the bound means the consumers are the limit and the bound is doing its job.
- "What about ordering?" Multiple consumers destroy input order. If the output must be ordered, tag items with an index and reassemble, or use one consumer.
Common mistakes
- One sentinel for many consumers, which hangs every consumer but one.
- An unhandled exception in the consumer loop, which silently reduces the worker count.
- A bound so large it never fills, which is an unbounded queue with extra steps.
- Returning from a consumer without calling task_done, which hangs any q.join().
Key takeaways
- One sentinel per consumer, queued in the producer's finally block so a producer failure still shuts down cleanly.
- Wrap the work in try/except inside the loop; an escaping exception kills a worker silently and looks like a slowdown.
- The bound is backpressure: small, a few times the consumer count. At a producer 3x faster than the consumers, an unbounded queue grows 20 MB/s.
- Instrument the queue's high-water mark; consistently at the bound means backpressure is working.