backpressure

Name

backpressure — backpressure — refuse work upstream so the buffer never overflows

Synopsis

backpressure [--window N] [--drop | --block] PRODUCER CONSUMER
in > out  ⇒  push back until  in == out

Description

backpressure regulates a producer against a slower consumer by propagating the consumer's incapacity backward along the pipe. When the buffer between them approaches capacity, the daemon emits a slow-down signal upstream; a compliant producer throttles its emission rate to match the drain rate. The buffer then holds near full and stops growing. This is the intended steady state, not a fault.

A producer that ignores the signal, or a pipe with no signal to send, degrades to the fallback policy: arrivals to a full buffer are dropped. backpressure does not prevent loss. It only offers the producer a chance to prevent it, and records who declined.

Options

--window N

Advertise room for N more items and no more. The producer may send exactly N, then must wait to be told again. Shrinks to zero on a full buffer.

--block

When the buffer is full, suspend the producer until space appears. Nothing is lost. Nothing moves either, which some callers report as a hang.

--drop

When the buffer is full, discard the arriving item and keep accepting. Latency stays flat. The data does not.

--optimistic

Assume the consumer will speed up. It will not. Retained for the sake of the hopeful.

--faster

Instruct the producer to push harder against a full buffer. Increases the drop count. Accepted, logged, blamed.

Exit Status

0

in == out; buffer steady; full and lossless

11

producer heeded the signal and throttled (SIGSLOW)

28

buffer full, no backpressure available; items dropped on the floor

75

consumer fell behind; lag climbing; the tank is reddening

History

"'Back pressure' is a fluid- and gas-dynamics term for pressure that opposes the intended flow — canonically the resistance an internal-combustion engine's exhaust system exerts against the outgoing gases. Computing inherited the mechanism intact: a Unix pipe whose 64 KB buffer is full simply blocks the writing process until the reader consumes, and if the reader has exited, the writer instead receives SIGPIPE and dies. TCP encoded the same idea as a number — the receive window of RFC 793 (1981) — which shrinks to zero when the receiver is saturated and is reopened by a window-update segment, the sender meanwhile firing zero-window probes to ask whether it may resume. Reactive Streams (2015, later folded into java.util.concurrent.Flow in JDK 9) turned the signal into an explicit request(n) call, making demand something the consumer states out loud."

Bugs

--block and --drop are mutually exclusive but both are frequently enabled by whoever configured the queue, whichever failure arrives first wins.

A producer marked compliant may still ignore SIGSLOW. There is no way to compel a firehose to respect a teacup.

The unread count on your personal buffer only increments. This is documented as working.

See Also

thrashing(1), livelock(8), quiescence(8). The living exhibit demonstrates the word in motion:

▸ operate backpressure

Author

"Maintained by The Pipe, which has an opinion about your speed and has been displaying it on the pressure bar the entire time."