Chapter 4: Why Systems Surprise Us
tis systems-thinking nonlinearity bounded-rationality delays
Status: Notes complete
Overview
Three truths frame this chapter: everything we know is a model, our models have strong congruence with the world, and our models fall far short of representing the world fully — hence constant surprise. Six structural causes explain why systems consistently defy our expectations.
1. Beguiling Events — Seeing Too Shallowly
Systems present themselves as a series of events. But events are just the tip of the iceberg.
Three levels of understanding:
| Level | Description | Explanatory Power |
|---|---|---|
| Event (shallowest) | “The stock market fell because the dollar rose.” | None — no prediction, no understanding |
| Behavior | Trends over time, patterns, historical context | Moderate |
| Structure (deepest) | Interlocking stocks, flows, and feedback loops | Full — reveals levers for change |
“System structure is the source of system behavior. System behavior reveals itself as a series of events over time.”
Systems thinkers constantly oscillate between structure (diagrams of stocks, flows, feedback) and behavior (time graphs).
Why behavior-based models fail: They find statistical links between flows, but flows only correlate because they’re governed by the same stocks. Change the underlying structure, and the correlations break. This is why econometric models predict short-term reasonably but fail long-term.
2. Linear Minds in a Nonlinear World
“Nonlinearity means that the act of playing the game has a way of changing the rules.”
Linear: cause produces proportional effect. Double the input → double the output.
Nonlinear: cause does not produce proportional effect. Twice the fertilizer may produce no yield increase, or decrease.
Examples of Nonlinearities
- Traffic: speed affected only slightly as density builds — then small additional density causes rapid speed collapse → traffic jam
- Soil erosion: proceeds for a long time with little crop yield effect — until topsoil hits root depth, then small further erosion → yield plummets
- Advertising: tasteful advertising builds interest; excessive advertising creates disgust
Why Nonlinearities Matter
- Confound the expectation that if a little helps, a lot helps more
- Change the relative strengths of feedback loops — the chief cause of shifting dominance
- Can flip a system from one behavioral mode to another entirely
Case: Spruce Budworm — Logging industry sprayed pesticides to suppress periodic outbreaks, killing natural predators too. Result: persistent near-outbreak conditions over larger areas. All relationships were nonlinear (reproduction vs. food supply, predator response vs. budworm density). Spraying created a volcanic state: always simmering, one condition away from eruption at unprecedented scale.
3. Nonexistent Boundaries
“There is no single, legitimate boundary to draw around a system.”
Diagrams use clouds to represent sources and sinks we’re choosing to ignore. Clouds are artificial mental-model constructs, not real boundaries.
The Boundary Problem
- Real boundaries don’t exist; everything is connected to everything else
- Greatest complexities arise at boundaries (where ecosystems meet, where cultures mix, where jurisdictions meet)
- Too narrow a boundary → surprised by effects you excluded (highways attract development → more traffic → same congestion)
- Too wide a boundary → overcomplication; piles of information that obscure the answers
“There are no separate systems. The world is a continuum. Where to draw a boundary around a system depends on the purpose of the discussion.”
4. Layers of Limits — Liebig’s Law of the Minimum
Systems surprise us because we think about single causes producing single effects. In reality, multiple inputs constrain any system simultaneously.
Liebig’s Law: It doesn’t matter how much nitrogen you have if phosphorus is the shortage. The most important input is the most limiting one at any given moment.
“At any given time, the input that is most important to a system is the one that is most limiting.”
Layers Change As Systems Grow
Growth itself depletes or enhances limits, shifting which factor becomes next-most-limiting:
- Salespeople generate orders faster than factory can produce → production capacity is limiting
- Expand capacity → hire fast, train too little → quality fails → labor skill is limiting
- Invest in training → quality improves → record-keeping system clogs → management bandwidth is limiting
Implication: No physical system can grow forever. Better to choose your limits than have them chosen for you.
5. Ubiquitous Delays
“We are surprised over and over again at how much time things take.”
Delays appear everywhere:
- Time from catching an infectious disease to diagnosis
- Time from pollution emission to detectable harm
- Gestation delays causing commodity price cycles (4-year pig cycles, 7-year cattle cycles)
- Social norm changes taking a generation
- Capital stock turnover taking 10-15+ years
Why delays matter: A feedback loop with a long delay will overshoot its goal, causing oscillations.
“When there are long delays in feedback loops, some sort of foresight is essential. To act only when a problem becomes obvious is to miss an important opportunity to solve the problem.”
Jay Forrester’s rule of thumb: Whatever delay people estimate, multiply by three.
6. Bounded Rationality
“Bounded rationality means that people make quite reasonable decisions based on the information they have. But they don’t have perfect information, especially about more distant parts of the system.”
We are not omniscient optimizers. We are “blundering satisficers” — finding a solution we can live with right now, changing only when forced to.
We Also Fail to Interpret Information Perfectly
- We misperceive risk
- We pay too much attention to recent events, too little to long-term history
- We discount the future at economically/ecologically irrational rates
- We don’t let in information that doesn’t fit our mental models
The Great Insight
If you were placed in someone else’s position in the system, you would behave as they do:
- As a manager, you’d see labor as a cost to minimize
- As a very poor person, you’d see the rationality of having many children
- As a fisherman with a mortgage and imperfect information about fish stocks, you’d overfish
“Taking out one individual from a position of bounded rationality and putting in another person is not likely to make much difference.”
The Way Out
Redesign the system to improve the information, incentives, disincentives, goals, stresses, and constraints impinging on actors.
Dutch electric meters: Homes with meters in the front hall used 30% less electricity than identical homes with basement meters. The only difference was information visibility — even slight enlargement of bounded rationality can produce large behavioral changes.
Summary
| Source of Surprise | Nature | Antidote |
|---|---|---|
| Event-level thinking | Focusing on outputs, not structure | Look for structure; trace feedback loops |
| Linear thinking | Expecting proportional response | Expect nonlinearity; watch for thresholds |
| False boundaries | Ignoring relevant connections | Draw boundaries appropriate to the question |
| Layers of limits | Ignoring the binding constraint | Identify the most limiting factor now; anticipate next ones |
| Delays | Underestimating time lags | Multiply delay estimates by ~3; build foresight |
| Bounded rationality | Acting locally without full information | Redesign information flows, incentives, system structure |
Last Updated: 2026-05-30