Fix PathConflictResolver OOM on dense dependency graphs - #2153
Conversation
gnodet-bot
left a comment
There was a problem hiding this comment.
The fix correctly addresses the OOM by bounding Path creation from exponential to O(graph-edges). Correctness is maintained — addChildren always inserts every child's Path into the partition regardless of expansion, so winner selection sees all occurrences. Three issues to address.
This review was generated by an AI agent, Hermès on behalf of @gnodet.
71451f0 to
0dd25a6
Compare
gnodet-bot
left a comment
There was a problem hiding this comment.
Re-review after new commits.
Previous findings — status:
- ✅ Javadoc accuracy (was: "at most once") — Fixed. All three locations (class Javadoc,
expandedNodesfield Javadoc,gatherCRNodesmethod Javadoc) now use accurate bounded-expansion language. - ✅
expandedNodescapacity hint — Fixed. Nownew IdentityHashMap<>()with no capacity argument. ⚠️ Regression test — Partially addressed. Test added, but see inline comment.
This review was generated by an AI agent, Hermès on behalf of @gnodet.
0dd25a6 to
abce9c1
Compare
gnodet-bot
left a comment
There was a problem hiding this comment.
Re-review after new commits.
Previous findings — status:
- ✅ Javadoc accuracy — Fixed in all three locations (class Javadoc,
expandedNodesfield Javadoc,gatherCRNodesmethod Javadoc). Language now accurately describes bounded re-expansion. - ✅
expandedNodescapacity hint — Fixed. Nownew IdentityHashMap<>()with default capacity. ⚠️ Regression test — Topology fixed (hubs now have children, guard is exercised), but test parameters are too small to detect the OOM regression. See inline comment.
This review was generated by an AI agent, Hermès on behalf of @gnodet.
In highly connected graphs (e.g. a 813-module reactor where each module depends on ~9 others), PathConflictResolver created an exponential number of Path objects: each DependencyNode reachable via N different parent paths was expanded N times, causing its entire subtree to be traversed N times. This resulted in OutOfMemoryError in gatherCRNodes/addChildren. Fix: track which DependencyNode instances have already been expanded in an IdentityHashMap<DependencyNode, Integer> (mapping to expansion depth). When the same DependencyNode is encountered again via a different parent path, a Path entry is still created (so all occurrences appear in the conflict partition for winner selection), but its subtree is not re- traversed. If the same node is later reached at a shallower depth, it is re-expanded at that depth, updating the recorded minimum. Also eliminates the per-Path HashSet<String> copy used for O(1) cycle detection: since the expansion deduplication already bounds total path count to O(graph edges), the parent-chain walk for hasConflictIdOnPathToRoot (O(depth), bounded by ~30 in practice) is both sufficient and cheaper than copying a growing HashSet on every Path construction. Verified: the 813-module reproducer (server_subset.zip) that previously caused OutOfMemoryError with -Xmx512m now completes successfully. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
abce9c1 to
2a5b081
Compare
gnodet-bot
left a comment
There was a problem hiding this comment.
Re-review after new commits.
Previous findings — status:
- ✅ Javadoc accuracy — Fixed in all three locations.
- ✅
expandedNodescapacity hint — Fixed. - ✅ Regression test — Fixed. Test now uses
M=N=K=100withassertTimeout(Duration.ofSeconds(5)), which provides a genuine regression guard: without the fix, the 1,010,000Pathobject allocation is slow enough that the 5-second bound fires even before OOM. Correctness assertions (hubCount == N,subHubCount == K) are sound — hub and sub-hub nodes all have distinct artifactIds, so no conflicts remove them, andTreeDependencyVisitorcounts each identity-unique instance once. The math in the comment (M×N + M×N×K = 1,010,000 Path objects) is accurate.
No further issues. The fix is correct and the test locks it in.
This review was generated by an AI agent, Hermès on behalf of @gnodet.
Problem
PathConflictResolverruns out of memory (OutOfMemoryError: Java heap space) on highly connected dependency graphs, such as a 813-module reactor where each module depends on ~9 others.Reproduced with
-Xmx512m:Workaround:
-Daether.conflictResolver.impl=classicRoot Cause
gatherCRNodesperforms a BFS/DFS traversal that creates a newPathobject for every edge in the expanded dependency tree — not just one perDependencyNode. When the same node is reachable via N different parent paths, its entire subtree is traversed N times, leading to exponentialPathallocation.For 813 modules × ~9 avg deps, this produces millions of
Pathobjects instead of the ~7,500 that actually represent unique graph edges.Fix
Two complementary changes:
1. Expansion deduplication (the OOM fix):
Track which
DependencyNodeinstances have already been expanded in anIdentityHashMap<DependencyNode, Integer>(mapped to expansion depth). When the sameDependencyNodeis encountered again via a different parent path, aPathentry is still created for it (so all occurrences appear in the conflict partition for winner selection), but its subtree is not re-traversed. If the same node is later reached at a shallower depth, it is re-expanded and the recorded minimum is updated.2. Remove per-Path
HashSetcopy (CPU/memory fix):The
HashSet<String> conflictIdsOnPaththat was copied on everyPathconstruction is dropped. Since expansion deduplication already bounds total path count to O(graph edges), the O(depth) parent-chain walk forhasConflictIdOnPathToRootis sufficient and cheaper. This also eliminates the per-node allocation that was identified as a JFR hotspot in the prior PR #2075.Testing
maven-resolver-utiltests passdenseGraphDoesNotOomadded: 3-level shared graph (100×100×100), would create ~1MPathobjects and OOM without the fix, completes in <100ms with itpathandclassicresolvers produce identical results on the reproducerPerformance — 813-module reproducer (813 modules × 9 deps = 7,317 edges)
Benchmarked on the exact reproducer topology (circular dependency graph, all nodes shared):
Minimum heap to complete without OOM:
PathConflictResolver(fixed)ClassicConflictResolverFirst-run latency (cold JIT, as in a real build):
PathConflictResolver(fixed)ClassicConflictResolverWarmed latency (JIT steady state):
PathConflictResolver(fixed)ClassicConflictResolverThe fixed
PathConflictResolveris ×21 faster cold and ×30 faster warmed thanClassicConflictResolver, while also requiring 40% less heap. The classic resolver's 5 s cold time is dominated by GC pressure when the heap is near its minimum: it allocates heavily during traversal and spends most of the first run collecting garbage.Hermes Agent (Claude Sonnet 4.6) on behalf of Guillaume Nodet