Chapter 21: Dependency Management
seg dependency-management semver live-at-head compatibility open-source
Status: Notes complete
Overview
Chapter 21 tackles one of the most persistent and underappreciated problems in software engineering: managing the dependencies your code takes on other code. Unlike most engineering problems, dependency management is fundamentally a social and coordination problem disguised as a technical one. The difficulty is not that the tools are bad — it is that the participants in the ecosystem have conflicting requirements, different time horizons, different incentives, and incomplete information about each other.
The chapter opens with a stark contrast: Google’s internal dependency management (a monorepo with Live at Head semantics) works reasonably well, while external dependency management is described as “a lurking crisis.” This gap is not accidental. Internal systems allow Google to enforce compatibility requirements, run automated tests across dependents, and coordinate upgrades at scale. External open-source ecosystems have none of these properties, and the tooling that has evolved (primarily Semantic Versioning) is an incomplete substitute.
The authors argue that SemVer, despite being nearly universal in the open-source world, is fundamentally built on promises that library maintainers frequently cannot keep and that version constraints cannot verify. The chapter ends with a sobering conclusion: dependency management at scale in the open-source ecosystem is an unsolved problem, and current best practices are approximations that work until they don’t.
Core Concepts
Dependency: Code your code relies on that is not under your direct control. This includes open-source libraries, internal shared libraries, language runtimes, compilers, and operating system interfaces. The key property is that the dependency can change independently of your code.
Diamond Dependency: The situation where project A depends on libraries B and C, and both B and C depend on library D — but on incompatible versions of D. Most build systems can only include one version of D, so either B or C (or both) will be broken. The diamond shape gives this its name.
Semantic Versioning (SemVer): A versioning convention in which version numbers have the form MAJOR.MINOR.PATCH, where MAJOR changes indicate breaking changes, MINOR changes indicate backward-compatible new features, and PATCH changes indicate backward-compatible bug fixes.
Live at Head: A dependency management strategy in which all consumers of a library always use the latest version. Requires library maintainers to ensure backward compatibility with all consumers before releasing, or to update consumers as part of the release. Google’s internal model approximates this.
Minimum Version Selection (MVS): A dependency resolution algorithm, used by Go modules, that selects the minimum version of each dependency that satisfies all constraints, rather than the maximum. Produces more predictable and reproducible builds than algorithms that select the latest-compatible version.
Compatibility promise: A library maintainer’s commitment about what changes they will not make between versions — specifically, what changes will not break existing callers. SemVer is an attempt to encode compatibility promises in version numbers.
Why Dependency Management Is Hard
The authors identify a fundamental structural problem: dependency management requires coordinating across trust boundaries. Your code depends on library B, whose authors have no direct relationship with your codebase, no visibility into how you use their API, and no obligation to preserve the exact behaviors you depend upon.
Three factors combine to make this hard:
1. Conflicting Requirements
Different users of a library have different requirements. One user needs the library to be stable and backward-compatible forever. Another user needs the latest security patches. A third needs new features. A fourth needs fixes for bugs that only affect their platform. No library maintainer can satisfy all of these simultaneously.
2. The Diamond Dependency Problem
Your Application
/ \
Library B Library C
\ /
Library D v1.0 ← B requires this
Library D v2.0 ← C requires this (breaking change)
If Library D v2.0 introduced breaking changes from v1.0, there is no single version of D that satisfies both B and C. The dependency graph has become unsatisfiable. Solutions include:
- Vendoring: Include multiple copies of D at different versions (increases binary size, can cause symbol conflicts)
- Waiting for B or C to upgrade: Requires coordinating with maintainers who may be unresponsive
- Forking: Maintaining your own patched fork of B or C (ongoing maintenance burden)
- Avoiding the dependency entirely: Not always feasible
The diamond dependency problem is not hypothetical — it is why large-scale projects with many transitive dependencies regularly hit dependency conflicts.
3. Hyrum’s Law Applies to Dependencies
Just as Hyrum’s Law states that users will depend on any observable behavior of an API regardless of what the documentation says, the same applies to dependencies. Users depend not just on documented interfaces but on:
- The exact output format of a function (not just its return type)
- The exception types thrown in edge cases
- The exact timing behavior of asynchronous operations
- The memory layout of objects
- Bug-compatible behavior (bugs that downstream code accidentally relies on)
When a library maintainer changes any of these — even to fix a bug — they break callers who depended on the old behavior.
Importing Dependencies
The Decision to Import
The chapter argues that the decision to take a dependency is not free. Each new dependency is a commitment:
- Your build system must be able to resolve the dependency
- Every future version of the dependency must remain compatible with your usage, or you must be willing to upgrade
- Security vulnerabilities in the dependency become your vulnerabilities
- The dependency’s own transitive dependencies become your transitive dependencies
The “one small utility function” trap: Importing an entire library to use one utility function brings in all the library’s transitive dependencies, all its future maintenance burden, and all its potential security vulnerabilities. The cost is often disproportionate to the benefit.
Compatibility Promises
When importing a dependency, you are implicitly relying on the maintainer’s compatibility promise. The chapter identifies a spectrum of compatibility promises:
| Commitment Level | Description | Example |
|---|---|---|
| No promise | API may change at any time without notice | Internal experimental APIs |
| Best-effort | Maintainer tries to avoid breaking changes but makes no guarantee | Many small open-source projects |
| SemVer | Breaking changes only on MAJOR version bumps | Most modern open-source libraries |
| Formal deprecation policy | Breaking changes announced with a migration period | Google’s public APIs, Java SE |
| Never break | API is frozen; no breaking changes ever | POSIX, some C standard library functions |
Most open-source libraries promise SemVer compatibility. The question is how well that promise is kept.
How Google Handles Importing
Google’s internal approach to external dependencies (the “third-party” model) is notably conservative:
- External dependencies are imported into the monorepo at a specific version
- They are updated deliberately, with testing against all internal consumers
- Only one version of any external library exists in the monorepo at a time (the One-Version Rule)
- Adding a new external dependency requires explicit approval
This model sacrifices flexibility (teams cannot independently choose their own library versions) for predictability (there are no diamond dependency problems internally).
Dependency Management Models
1. Static Dependency Model (Nothing Changes)
The simplest model: pin every dependency to a specific version and never update. This is common in embedded systems and safety-critical software.
Works when:
- The system has a known, finite lifetime
- The environment is controlled and does not change
- Security updates are not required (or are obtained through other means)
Fails when:
- Security vulnerabilities are discovered in pinned versions
- The operating environment changes (new OS, new hardware)
- Functionality from newer versions is needed
- The pinned version reaches end-of-life and stops receiving patches
Real-world example: A long-running production system on a frozen Linux kernel with locked library versions. Works well until CVEs force emergency patches, at which point the cost of updating dependencies that have diverged over years is enormous.
2. Semantic Versioning (SemVer)
SemVer is the dominant model in the open-source ecosystem. Libraries declare a version number MAJOR.MINOR.PATCH, where:
- MAJOR: Incompatible API changes (breaking changes)
- MINOR: Backward-compatible new functionality
- PATCH: Backward-compatible bug fixes
Consumers specify version constraints (e.g., ^1.2.0 meaning ”>=1.2.0 and <2.0.0”) and dependency resolution tools find a set of versions satisfying all constraints.
The appeal of SemVer: It encodes the library maintainer’s compatibility promise in a machine-readable form. Tools can automatically check whether a proposed dependency set is theoretically compatible.
The problems with SemVer (covered in depth below).
3. Bundled Distribution Models
Some ecosystems manage dependencies by bundling a curated, tested set of library versions together into a distribution. Users consume the distribution rather than individual libraries.
Examples:
- Linux distributions (e.g., Ubuntu packages a tested, compatible set of libraries)
- Anaconda/conda in the Python ecosystem (packages a tested scientific computing stack)
- Platform-managed dependencies (e.g., Java EE application servers bundling compatible library versions)
Benefits: The distribution maintainer has tested that all the bundled libraries are compatible with each other. Users get a known-good combination.
Drawbacks: Individual users cannot easily upgrade a single library within the bundle. Users who need library versions outside the bundle’s constraints must either patch the bundle or manage dependencies manually.
4. Live at Head
The most demanding model: all consumers of all libraries always use the latest version. No version constraints; just “HEAD.”
How it works at Google:
- All code — both internal libraries and their consumers — lives in the same monorepo
- When a library maintainer makes a change, they are responsible for updating all consumers in the same commit (or shortly after)
- Automated tests run across all dependents before a change is submitted
- If a change breaks any dependent, it cannot be submitted until either the change or the dependent is fixed
Requirements for Live at Head to work:
- A monorepo (or equivalent global view of all code)
- Comprehensive test coverage of all dependents
- Tooling to automate large-scale changes across the codebase (see Chapter 22)
- A culture that treats “breaking a dependent” as an unacceptable outcome
- Enough engineering resources to update dependents as part of the change
What Live at Head eliminates: Diamond dependencies (there is only one version), version constraint negotiation, and the accumulation of version skew across the codebase.
What Live at Head requires: Much more discipline and tooling from library maintainers, and a centralized codebase structure.
The Limitations of SemVer
SemVer Might Overconstrain
SemVer-based dependency resolution can fail to find a valid dependency set even when one theoretically exists. If library B requires D >= 1.0.0, < 2.0.0 and library C requires D >= 1.5.0, < 2.0.0, both can be satisfied by D 1.5.x. But if B has an overly conservative constraint (perhaps it was written before D 1.5 was released) of D >= 1.0.0, < 1.3.0, the resolution fails even though B actually works fine with D 1.5.
Overconstrained dependency graphs are a real-world problem: maintainers set conservative upper bounds to avoid untested version combinations, but these bounds prevent upgrades even when the newer version would work fine.
SemVer Might Overpromise
Conversely, a library maintainer might declare a release as a MINOR version bump (no breaking changes) when it actually breaks some users. This can happen because:
- Hyrum’s Law: Users depend on behaviors the maintainer did not consider part of the public API
- Incomplete testing: The maintainer did not test all usage patterns
- Platform differences: A change is backward-compatible on one OS but not another
- Performance changes: A change is semantically correct but breaks callers that depended on specific performance characteristics
When SemVer overpromises, downstream tooling (which trusts the version number) will happily upgrade to a “non-breaking” version and introduce regressions.
Motivations Behind SemVer
The authors are clear-eyed about why SemVer exists and why it is imperfect:
SemVer is a social contract, not a technical guarantee. It communicates intent (“I tried not to break you”) rather than a verified property (“I have proven this does not break you”). The version number is the maintainer’s honest assessment of their changes, but:
- Maintainers have imperfect knowledge of how their library is used
- “Breaking” is defined relative to what callers depend on (Hyrum’s Law), which the maintainer cannot fully know
- Incentives are misaligned: the cost of getting the version number wrong is borne by downstream users, not the maintainer
The fundamental problem: SemVer is a lossy encoding of compatibility information. A single bit (MAJOR vs. MINOR) cannot capture the full complexity of which callers will be affected by a change and which will not.
Minimum Version Selection (MVS)
Go modules use MVS instead of the “maximum compatible version” algorithm common in other package managers (npm, pip, cargo). The key difference:
| Algorithm | Behavior | Effect |
|---|---|---|
| Maximum compatible version | Select the latest version satisfying constraints | Users get newer versions automatically; more likely to encounter bugs in new versions |
| Minimum Version Selection (MVS) | Select the minimum version satisfying constraints | Users get exactly what was tested; builds are more reproducible |
MVS rationale: If a project specifies D >= 1.2.0, it has tested with 1.2.0. Selecting 1.2.0 gives a known-good result. Selecting 1.7.0 (the latest) gives an untested result that might have regressions.
MVS trade-off: Users must explicitly opt into version upgrades. They will not automatically receive bug fixes or security patches that were released as MINOR or PATCH versions. This makes MVS more conservative and reproducible but requires more active dependency management.
MVS does not solve the diamond dependency problem — it just makes the algorithm more predictable when the problem does not occur. It also does not address the fundamental issue that version constraints are unverifiable promises.
Does SemVer Work? An Honest Assessment
The chapter provides an unusually candid assessment: SemVer works reasonably well in small dependency graphs with conscientious maintainers, but degrades as graphs become larger and more complex.
Arguments in favor: SemVer has enabled the npm/cargo/pip ecosystems to function at enormous scale. Most SemVer promises are kept most of the time. The alternative (no versioning convention at all) is clearly worse.
Arguments against:
- Large transitive dependency graphs regularly produce unsatisfiable constraint sets
- “Breaking changes” are routinely released as MINOR versions (accidentally or deliberately)
- The ecosystem creates perverse incentives: libraries avoid MAJOR bumps (which scare users away) by cramming breaking changes into MINOR releases or avoiding the change entirely
- Version constraint ranges quickly become out of date as maintainers add unnecessary upper bounds
- The resulting “dependency hell” (where upgrading one library requires cascading upgrades of many others) is a common and costly engineering experience
The underlying problem SemVer cannot solve: SemVer operates on the assumption that compatibility is a binary property — a version either is or is not compatible with a given dependency. In reality, compatibility is a function of how the dependency is used. Version numbers cannot encode this.
Dependency Management with Infinite Resources
The authors pose a thought experiment: if you had infinite engineering resources, how would you manage dependencies?
The answer reveals what current practices are approximating:
-
Test all combinations: Before accepting any dependency change, run your full test suite (and all your dependents’ test suites) against the new version. If anything breaks, reject the change or fix the breakage first.
-
Verify compatibility claims: Rather than trusting a maintainer’s SemVer declaration, verify it by running automated compatibility tests against all known consumers.
-
Update all consumers simultaneously: When a breaking change is unavoidable, update all consumers in the same operation that introduces the change. No consumer is ever left in a broken state.
This is essentially what Google does internally with its monorepo and automated large-scale change tooling. The gap between this ideal and what open-source ecosystems can achieve is a consequence of the trust boundary problem: you cannot run other organizations’ test suites, you cannot update other organizations’ code, and you cannot require other organizations to keep pace with your changes.
Exporting Dependencies: Responsibilities of Library Maintainers
Most of the chapter’s practical advice is directed at organizations that export (publish) libraries, not just consume them.
The Core Obligation
When you publish a library that others depend on, you are accepting obligations:
- To maintain your API in a way that does not break callers unnecessarily
- To communicate breaking changes through version numbers (SemVer) or formal deprecation notices
- To provide migration guidance when breaking changes are unavoidable
- To consider the full observable behavior of your API, not just the documented interface
The Challenge of Compatible Releases
Making a truly compatible release is harder than it appears. Compatibility is determined by what your callers actually depend on (Hyrum’s Law), not by what you documented. A “non-breaking” change that modifies exception types, logging output, or performance characteristics can break callers.
Practical advice from the chapter:
- Run tests against known consumers before releasing
- Use deprecation warnings well before removing features
- Provide automated migration tooling for breaking changes when possible
- Understand that your callers will depend on more of your behavior than you expect
- When in doubt, treat a change as breaking
Avoiding Unnecessary Breaking Changes
The authors make a distinction between necessary and unnecessary breaking changes. Necessary breaking changes include security fixes, fundamental design corrections, and feature additions that cannot be made backward-compatible. Unnecessary breaking changes are the result of poor API design, insufficient testing, or insufficient consideration of how the API is used.
Library maintainers should aggressively avoid unnecessary breaking changes, which requires:
- Designing APIs with stability in mind from the start
- Being conservative about what behaviors are observable and therefore part of the contract
- Using compatibility test suites to detect regressions before release
- Maintaining long-lived branches for major versions when breaking changes are unavoidable
The Responsibility of Popularity
The more widely used a library is, the more conservative its maintainers must be. A breaking change in a widely used library propagates across the entire ecosystem that depends on it, potentially requiring thousands of downstream projects to update. The asymmetry is stark: the benefit of the change accrues to the maintainer (cleaner code, better API), while the cost is distributed across all dependents.
TL;DRs
(Faithful reproduction from the book’s end-of-chapter TL;DR section)
- Dependency management is the art of managing your code’s dependencies over time and at scale.
- Every dependency you take on comes with associated requirements (correctness, security, compatibility) that are not necessarily in your control.
- Understanding what happens when dependencies break is important.
- SemVer is a lossy encoding of compatibility information. It attempts to encode expectations about compatibility in a machine-readable format, but those expectations are imperfect and frequently wrong.
- There is a fundamental difference between testing against a dependency and having that dependency tested against you.
- The diamond dependency problem is a fundamental challenge in dependency management, and current solutions are imperfect approximations.
- Live at Head is a policy that eliminates many dependency management problems but requires significant infrastructure and cultural investment.
- Minimum Version Selection (MVS) produces more reproducible builds by selecting the minimum satisfying version rather than the maximum.
- When exporting dependencies, you are accepting obligations to your consumers that extend beyond the documented API.
- The open-source dependency management ecosystem is in a precarious state: it works well enough until it doesn’t, and large-scale dependency graphs are frequently in a state of latent crisis.
Key Takeaways
- Dependency management is a coordination problem — the difficulty is not technical but social: it requires coordination across trust boundaries between parties with different requirements, incentives, and information.
- The diamond dependency problem is fundamental — when two dependencies require incompatible versions of a shared dependency, build systems have no clean solution; workarounds (vendoring, forking) all carry significant costs.
- SemVer is a social contract, not a technical guarantee — version numbers communicate a maintainer’s intent about compatibility but cannot be mechanically verified; Hyrum’s Law ensures that “non-breaking” changes will break some callers.
- Live at Head eliminates many dependency problems but requires a monorepo, comprehensive testing, automated large-scale change tooling, and a culture of treating dependent breakage as unacceptable.
- Minimum Version Selection (MVS) prioritizes reproducibility over automatic access to new fixes — it selects the minimum satisfying version rather than the latest compatible, making builds more predictable but requiring explicit version upgrades.
- Importing a dependency is a long-term commitment — it brings in transitive dependencies, security vulnerabilities, future maintenance burden, and a dependency on the maintainer’s compatibility discipline.
- Library maintainers bear obligations to their consumers that extend beyond the documented API to include all observable behaviors; breaking callers who relied on undocumented behavior is still breaking them.
- SemVer overconstrain and overpromise are both real failure modes: overly conservative version bounds prevent valid upgrades; optimistic version declarations introduce unexpected breakage.
- Dependency management with infinite resources (test all combinations, update all consumers simultaneously) reveals what current practices approximate; the gap is a consequence of the trust boundary problem in open-source ecosystems.
- Popular libraries carry asymmetric responsibility — the benefits of breaking changes accrue to the maintainer while the costs propagate across all dependents; popularity requires increased conservatism about compatibility.
Related Resources
- ch16-version-control — The monorepo model that enables Live at Head dependency management
- ch22-large-scale-changes — The tooling that makes it feasible to update all consumers when a dependency breaks them
- ch15-deprecation — How to retire APIs and libraries without stranding dependents
- DDIA Chapter 4 — Encoding and Evolution: parallel treatment of backward/forward compatibility at the data layer
Last Updated: 2026-06-02