Sitelet https://github.com/apache/maven-resolver/pull/2153
Skip to content

Fix PathConflictResolver OOM on dense dependency graphs - #2153

Merged
cstamas merged 1 commit into
apache:masterfrom
gnodet:fix/pcr-oom-dense-graph
Sep 26, 2026
Merged

cstamas merged 1 commit into
apache:masterfrom
gnodet:fix/pcr-oom-dense-graph

Conversation

@gnodet

@gnodet gnodet commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem

PathConflictResolver runs out of memory (OutOfMemoryError: Java heap space) on highly connected dependency graphs, such as a 813-module reactor where each module depends on ~9 others.

Reproduced with -Xmx512m:

java.lang.OutOfMemoryError: Java heap space
    at PathConflictResolver$Path.addChildren(PathConflictResolver.java:699)
    at PathConflictResolver$State.gatherCRNodes(PathConflictResolver.java:363)

Workaround: -Daether.conflictResolver.impl=classic

Root Cause

gatherCRNodes performs a BFS/DFS traversal that creates a new Path object for every edge in the expanded dependency tree — not just one per DependencyNode. When the same node is reachable via N different parent paths, its entire subtree is traversed N times, leading to exponential Path allocation.

For 813 modules × ~9 avg deps, this produces millions of Path objects instead of the ~7,500 that actually represent unique graph edges.

Fix

Two complementary changes:

1. Expansion deduplication (the OOM fix):
Track which DependencyNode instances have already been expanded in an IdentityHashMap<DependencyNode, Integer> (mapped to expansion depth). When the same DependencyNode is encountered again via a different parent path, a Path entry is still created for it (so all occurrences appear in the conflict partition for winner selection), but its subtree is not re-traversed. If the same node is later reached at a shallower depth, it is re-expanded and the recorded minimum is updated.

2. Remove per-Path HashSet copy (CPU/memory fix):
The HashSet<String> conflictIdsOnPath that was copied on every Path construction is dropped. Since expansion deduplication already bounds total path count to O(graph edges), the O(depth) parent-chain walk for hasConflictIdOnPathToRoot is sufficient and cheaper. This also eliminates the per-node allocation that was identified as a JFR hotspot in the prior PR #2075.

Testing

  • All 463 maven-resolver-util tests pass
  • Regression test denseGraphDoesNotOom added: 3-level shared graph (100×100×100), would create ~1M Path objects and OOM without the fix, completes in <100ms with it
  • Both path and classic resolvers produce identical results on the reproducer

Performance — 813-module reproducer (813 modules × 9 deps = 7,317 edges)

Benchmarked on the exact reproducer topology (circular dependency graph, all nodes shared):

Minimum heap to complete without OOM:

Resolver Min heap
PathConflictResolver (fixed) 28 MB
ClassicConflictResolver 48 MB

First-run latency (cold JIT, as in a real build):

Resolver Time
PathConflictResolver (fixed) ~260 ms
ClassicConflictResolver ~5,400 ms

Warmed latency (JIT steady state):

Resolver Time
PathConflictResolver (fixed) ~2–3 ms
ClassicConflictResolver ~50–90 ms

The fixed PathConflictResolver is ×21 faster cold and ×30 faster warmed than ClassicConflictResolver, while also requiring 40% less heap. The classic resolver's 5 s cold time is dominated by GC pressure when the heap is near its minimum: it allocates heavily during traversal and spends most of the first run collecting garbage.

Hermes Agent (Claude Sonnet 4.6) on behalf of Guillaume Nodet

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fix correctly addresses the OOM by bounding Path creation from exponential to O(graph-edges). Correctness is maintained — addChildren always inserts every child's Path into the partition regardless of expansion, so winner selection sees all occurrences. Three issues to address.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

@gnodet
gnodet force-pushed the fix/pcr-oom-dense-graph branch from 71451f0 to 0dd25a6 Compare September 22, 2026 12:16

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after new commits.

Previous findings — status:

  1. ✅ Javadoc accuracy (was: "at most once") — Fixed. All three locations (class Javadoc, expandedNodes field Javadoc, gatherCRNodes method Javadoc) now use accurate bounded-expansion language.
  2. ✅ expandedNodes capacity hint — Fixed. Now new IdentityHashMap<>() with no capacity argument.
  3. ⚠️ Regression test — Partially addressed. Test added, but see inline comment.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

@gnodet
gnodet force-pushed the fix/pcr-oom-dense-graph branch from 0dd25a6 to abce9c1 Compare September 22, 2026 12:39

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after new commits.

Previous findings — status:

  1. ✅ Javadoc accuracy — Fixed in all three locations (class Javadoc, expandedNodes field Javadoc, gatherCRNodes method Javadoc). Language now accurately describes bounded re-expansion.
  2. ✅ expandedNodes capacity hint — Fixed. Now new IdentityHashMap<>() with default capacity.
  3. ⚠️ Regression test — Topology fixed (hubs now have children, guard is exercised), but test parameters are too small to detect the OOM regression. See inline comment.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

In highly connected graphs (e.g. a 813-module reactor where each module
depends on ~9 others), PathConflictResolver created an exponential number
of Path objects: each DependencyNode reachable via N different parent paths
was expanded N times, causing its entire subtree to be traversed N times.
This resulted in OutOfMemoryError in gatherCRNodes/addChildren.

Fix: track which DependencyNode instances have already been expanded in
an IdentityHashMap<DependencyNode, Integer> (mapping to expansion depth).
When the same DependencyNode is encountered again via a different parent
path, a Path entry is still created (so all occurrences appear in the
conflict partition for winner selection), but its subtree is not re-
traversed. If the same node is later reached at a shallower depth, it is
re-expanded at that depth, updating the recorded minimum.

Also eliminates the per-Path HashSet<String> copy used for O(1) cycle
detection: since the expansion deduplication already bounds total path
count to O(graph edges), the parent-chain walk for hasConflictIdOnPathToRoot
(O(depth), bounded by ~30 in practice) is both sufficient and cheaper than
copying a growing HashSet on every Path construction.

Verified: the 813-module reproducer (server_subset.zip) that previously
caused OutOfMemoryError with -Xmx512m now completes successfully.

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@gnodet
gnodet force-pushed the fix/pcr-oom-dense-graph branch from abce9c1 to 2a5b081 Compare September 22, 2026 12:58
@gnodet
gnodet requested a review from cstamas September 22, 2026 13:03

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review after new commits.

Previous findings — status:

  1. ✅ Javadoc accuracy — Fixed in all three locations.
  2. ✅ expandedNodes capacity hint — Fixed.
  3. ✅ Regression test — Fixed. Test now uses M=N=K=100 with assertTimeout(Duration.ofSeconds(5)), which provides a genuine regression guard: without the fix, the 1,010,000 Path object allocation is slow enough that the 5-second bound fires even before OOM. Correctness assertions (hubCount == N, subHubCount == K) are sound — hub and sub-hub nodes all have distinct artifactIds, so no conflicts remove them, and TreeDependencyVisitor counts each identity-unique instance once. The math in the comment (M×N + M×N×K = 1,010,000 Path objects) is accurate.

No further issues. The fix is correct and the test locks it in.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants