[MNG-8507] Reduce allocation pressure in model building pipeline - #12540
Conversation
5ef63b4 to
d3f572c
Compare
There was a problem hiding this comment.
Pull request overview
This PR reduces intermediate immutable Model object allocations during effective model construction (Maven 4), primarily by improving generated merger/model templates and by switching several pipeline SPIs/implementations to a builder-passing pattern to avoid build-then-rebuild round-trips.
Changes:
- Optimizes Modello-generated templates (
model.vm,merger.vm) to avoid false-positive builder writes and reduce map/list copying duringbuild()and location computation. - Updates
ImmutableCollections.copy(Map)to recognize JDK immutable map implementations and skip redundant wrapping. - Refactors model-building pipeline components (injectors/normalizers/translators/assemblers/expanders) and their SPIs to write into a provided
Model.Builderinstead of returning rebuiltModelinstances.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/mdo/model.vm | Reduces redundant collection copying and optimizes location map merging. |
| src/mdo/merger.vm | Adds identity/value guards to reduce unnecessary builder field writes during merges. |
| src/mdo/java/ImmutableCollections.java | Skips copying for JDK immutable map implementations. |
| pom.xml | Updates resolverVersion from snapshot to release. |
| impl/maven-testing/src/main/java/org/apache/maven/testing/plugin/MojoExtension.java | Adapts path alignment to builder-passing translator API. |
| impl/maven-impl/src/test/java/org/apache/maven/impl/model/DefaultModelPathTranslatorTest.java | Updates tests for builder-passing path translation API. |
| impl/maven-impl/src/test/java/org/apache/maven/impl/model/DefaultInheritanceAssemblerTest.java | Updates tests for builder-passing inheritance assembly API. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultProfileInjector.java | Implements builder-passing profile injection and adds no-op caching. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultPluginManagementInjector.java | Switches plugin management injection to builder-passing. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultModelPathTranslator.java | Switches path translation to builder-passing and avoids unchanged sub-object sets. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultModelNormalizer.java | Switches duplicate merge/default injection to builder-passing and avoids unchanged rebuilds. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultModelBuilder.java | Updates the effective-model pipeline to create/pass builders through each stage. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultInheritanceAssembler.java | Switches inheritance assembly to builder-passing and merges into provided builder. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/model/DefaultDependencyManagementInjector.java | Switches dependency management injection to builder-passing. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/DefaultPluginConfigurationExpander.java | Switches plugin configuration expansion to builder-passing and avoids unchanged sets. |
| impl/maven-impl/src/main/java/org/apache/maven/impl/DefaultModelUrlNormalizer.java | Switches URL normalization to builder-passing. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/ProfileInjector.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/PluginManagementInjector.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/PluginConfigurationExpander.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/ModelUrlNormalizer.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/ModelPathTranslator.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/ModelNormalizer.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/InheritanceAssembler.java | Updates SPI to accept Model.Builder. |
| api/maven-api-spi/src/main/java/org/apache/maven/api/services/model/DependencyManagementInjector.java | Updates SPI to accept Model.Builder. |
Comments suppressed due to low confidence (2)
impl/maven-impl/src/main/java/org/apache/maven/impl/DefaultModelUrlNormalizer.java:63
- The SCM URLs are normalized and written via a newly-built
Scmeven when normalization is a no-op. Because the model builders use identity checks for their short-circuit, writing a value-equal but different String instance can force unnecessary immutable copies. Only rebuild/setscmwhen at least one normalized field actually changes (including value-equality).
Scm scm = model.getScm();
if (scm != null) {
builder.scm(Scm.newBuilder(scm)
.url(/sitelet?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmaven%2Fpull%2Fnormalize%28scm.getUrl%28)))
.connection(normalize(scm.getConnection()))
.developerConnection(normalize(scm.getDeveloperConnection()))
.build());
}
impl/maven-impl/src/main/java/org/apache/maven/impl/DefaultModelUrlNormalizer.java:71
- Site URL normalization is applied via
withSite(site.withurl(/sitelet?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmaven%2Fpull%2F...))without checking whether the normalized URL actually differs. If normalization returns a value-equal String (or no-op), this can still allocate new immutable objects. Guard the update so DistributionManagement/Site are only rebuilt when the URL truly changes (including value-equality).
DistributionManagement dist = model.getDistributionManagement();
if (dist != null) {
Site site = dist.getSite();
if (site != null) {
builder.distributionManagement(dist.withSite(site.withurl(/sitelet?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmaven%2Fpull%2Fnormalize%28site.getUrl%28)))));
}
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
|
|
||
| Model.Builder builder = Model.newBuilder(model); | ||
| builder.url(/sitelet?url=https%3A%2F%2Fgithub.com%2Fapache%2Fmaven%2Fpull%2Fnormalize%28model.getUrl%28))); |
gnodet
left a comment
There was a problem hiding this comment.
Solid performance optimization — the benchmark results (28% less GC pause time, 7.6% fewer GC events) demonstrate meaningful improvement. The shared-builder pattern is well-designed. A few observations on behavioral changes embedded in the refactoring:
-
Report plugin expansion bug fix: In
DefaultPluginConfigurationExpander, the old code calledexpandReport(reporting.getPlugins())but discarded the return value — report plugin configuration-to-report-set merging was silently broken. The new code correctly captures and applies the expanded report plugins. This is a genuine bug fix worth calling out explicitly in the PR description so testers are aware of the changed behavior. -
MojoExtension test model alignment: In
MojoExtension.beforeEach, the code changed from aligningtmodel(raw parsed POM) to aligningmodel(merged with defaults). Whentmodel == null, the old code stored null; the new code stores the aligned default model. Both changes look like intentional improvements, but the behavioral delta may affect test expectations. -
SPI interface changes: Seven SPI interfaces (
ProfileInjector,InheritanceAssembler, etc.) changed from returningModelto acceptingModel.Builderand returningvoid. This is source/binary-incompatible for any third-party implementations, though acceptable since Maven 4 is pre-release and these are all@since 4.0.0.
Minor notes:
- The
resolverVersionbump from2.0.21-SNAPSHOTto2.0.21inpom.xmlis unrelated to the optimization work. - The
ImmutableCollectionsJDK-internal class name detection is fail-safe (falls through to defensive copy), which is good.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
elharo
left a comment
There was a problem hiding this comment.
This is imnportant; ask me about handshoe sometime
| // Check if this is already an immutable JDK map (from Map.of(), Map.copyOf(), etc.) | ||
| // Those throw UnsupportedOperationException on put() and don't need copying | ||
| String className = map.getClass().getName(); | ||
| if (className.startsWith("java.util.ImmutableCollections$")) { |
There was a problem hiding this comment.
Is this guaranteed by the JDK or could the actual class names used here change in the future?
Is there any other way we could test this? Maybe addAll(Collections.emptyList()) and see if it throws or not.
There was a problem hiding this comment.
Good catch — the JDK internal class names are indeed not guaranteed and could change between versions. Rather than switching to a mutation-based test (try/catch on put() has its own downsides: performance overhead on the hot path, reliance on exception semantics), I've simply removed the check entirely. In practice, JDK Map.of()/Map.copyOf() maps rarely appear in the model builder — most maps are either our own AbstractImmutableMap (already short-circuited) or HashMap from merging. The wrapping cost for the occasional JDK immutable map is a single array copy of the entries, which is negligible.
gnodet
left a comment
There was a problem hiding this comment.
Re-review after new commits (83a37f7, 7adf18c, 2f1c0a5)
Looks great — the new commits cleanly address elharo's feedback (fragile JDK class name check removed) and the merger template false-positive fix (Objects.equals instead of reference check for String fields) is an important correctness improvement.
Two minor observations:
-
Timing log level (
DefaultModelBuilder.java:838): The[TIMING]log usesINFOlevel, which means every Maven 4 reactor build prints internal timing info. All other timing/diagnostic logging inDefaultModelBuilderusesDEBUG— consider aligning this one too, unless you want it visible in production builds. -
Minor cache note (
DefaultProfileInjector.java:155):doInjectSingleProfilealways returnstruefor non-null profiles, so the cache never short-circuits single-profile no-op injections. Not a correctness issue — just a minor missed optimization opportunity.
Overall this is a well-executed performance optimization with solid benchmark results.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
| } | ||
|
|
||
| long t2 = System.nanoTime(); | ||
| logger.info( |
There was a problem hiding this comment.
Suggestion: this timing log uses INFO level, but all other timing/diagnostic logging in this class uses DEBUG. Consider changing to logger.debug(...) to keep production build output clean — unless you specifically want this visible for users.
| logger.info( | |
| logger.debug( |
gnodet
left a comment
There was a problem hiding this comment.
Re-review after commit 357cf69 ("Short-circuit interpolation for strings without variable references")
Clean optimization — skipping the full substVars pipeline (HashSet allocation, delimiter scanning, unescape processing) for strings without $ is sound. The null guard is consistent with existing behavior, and the $ check correctly covers both ${...} placeholders and $__ escape markers.
Looks good. 👍
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
JFR tracing on Apache Camel (676 modules) identified four allocation
hotspots in the model building pipeline. This commit eliminates them:
1. TypeRegistryAdapter.get: cached lookup results in a ConcurrentHashMap
(was creating HashMap + DefaultType + PathType[] on every call)
→ 2,201 events → 2 events (99.9% reduction)
2. DefaultInterpolator.doSubstVars: reuse a single HashSet for cycle
detection across all strings in one model, instead of allocating a
new HashSet per interpolated string
→ 1,296 events → 203 events (84% reduction)
3. PropertiesAsMap.entrySet iterator: cast the underlying Entry directly
instead of wrapping it in a new anonymous Entry object; replace
stream-based size() with a simple loop
→ 2,790 events → 212 events (92% reduction)
Additional fast-path optimizations:
- MavenTransformer (transformer.vm): indexOf('$') short-circuit to skip
the interpolation callback chain for strings without variable refs
- DefaultInterpolator.unescape: early return when string contains neither
escape markers nor escape characters
- ReflectionValueExtractor: static constant for accessor prefix list
Total model building allocation: ~4 GB → ~3 GB (25% reduction).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
357cf69 to
60fec5f
Compare
- DefaultModelValidator: hoist scope computation out of per-dependency loop; avoids redundant InternalSession.from(), stream pipelines, and array allocations on every dependency (Camel: ~13k deps) - InternalSession.from(): add instanceof fast-path to skip string concat in error message on the success path - DefaultProjectBuilder: compute root directory once before the reactor loop instead of calling rootLocator.findRoot() per module (Camel: 676 calls → 1) JFR-verified: total TLAB from 2,646 MB → 2,054 MB (~22% reduction). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Objects.hash(a, b, c) allocates a new Object[] on every call. With MavenProject used as HashMap key in sorting and dedup streams across 676 modules, JFR showed 89 events / 25 MB from this alone. Inline the hash computation to eliminate the varargs array. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The indexOf('$') short-circuit in transformer.vm skipped calling the
transformer function for strings without '$'. This is correct for
interpolation (${...}), but breaks settings decryption which uses the
same SettingsTransformer to apply decryptFunction on encrypted passwords
like {BteqUEnqHecHM7MZfnj9FwLcYbdInWxou1C929Txa0A=}.
Move the fast-path from the shared transformer.vm template into
DefaultModelInterpolator as an anonymous subclass override, where it
only applies to interpolation — not to decryption or other uses of
the generated transformers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
elharo
left a comment
There was a problem hiding this comment.
test failing:
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.952 s -- in org.apache.maven.it.MavenIT0008SimplePluginTest
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] MavenITmng7038RootdirTest.testRootdir:69 project.rootDirectory ==> expected: </Users/runner/work/maven/maven/its/core-it-suite/target/test-classes/mng-7038-rootdir/module-a> but was: </Users/runner/work/maven/maven/its/core-it-suite/target/test-classes/mng-7038-rootdir>
gnodet
left a comment
There was a problem hiding this comment.
Re-review after 4 new commits + elharo's CHANGES_REQUESTED
🔴 Confirmed regression: rootDirectory hoisting
The rootLocator.findRoot() call was hoisted out of the per-module loop in DefaultProjectBuilder.java (~line 560), but the assumption that all reactor modules share the same root directory is incorrect. Maven 4 allows sub-modules to declare root="true" in their POM (e.g., mng-7038-rootdir/module-a/pom.xml), making findRoot() return different results per module.
Computing root once from the top-level POM overrides per-module roots, causing MavenITmng7038RootdirTest.testRootdir to fail on all 9 CI matrix combinations:
expected: <.../module-a> but was: <.../mng-7038-rootdir>
Fix: Revert this single hoisting — keep rootLocator.findRoot(pom.getParentFile().toPath()) inside the loop.
✅ Other optimizations look good
- Settings decryption fix (094f0b5): Correct — moves the
indexOf('$') < 0fast-path to the model interpolator, preserving{...}decryption in settings. - hashCode varargs avoidance (26b636f): Textbook-correct manual hash computation matching
Objects.hash()semantics. - Scope validation pre-computation, HashSet reuse, ConcurrentHashMap cache, and other micro-optimizations are all safe and well-crafted.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
…38RootdirTest) Sub-modules can declare root="true" in their POM, so findRoot() must be called per-module rather than once from the reactor root. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Good catch on the Both |
gnodet
left a comment
There was a problem hiding this comment.
Re-review: rootLocator regression fixed ✅
Commit 4a51d64d correctly reverts the rootLocator.findRoot() hoisting — it's back inside the per-module loop, computed from each module's pom.getParentFile().toPath(). This resolves the MavenITmng7038RootdirTest failure.
All other optimizations in this PR remain sound. Looks good to merge once CI confirms.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of gnodet
|
@gnodet Please assign appropriate label to PR according to the type of change. |
Summary
Reduces allocation pressure in the Maven 4 model building pipeline to improve performance toward parity with Maven 3 for
mvn <goal> -oon large reactors (e.g. Apache Camel, 676 modules).Profiled with JFR
jdk.ObjectAllocationInNewTLABevents on Apache Camel. Overall allocation reduction: ~49% (from ~4,000 MB to ~2,054 MB).Changes
Commit 1: Core allocation reductions (
60fec5f)transformer.vm— short-circuittransform(String): return strings without$immediately (skips ~90% of strings)model.vm— optimizecomputeLocations()to avoidEntry.copyOf()when locations are emptymodel.vm— skipImmutableCollections.copy()in constructors when the builder's collection is already the base valueImmutableCollections— recognize JDK unmodifiable maps/lists to avoid redundant wrappingDefaultModelNormalizer— replace unconditionalbuilder.build()with identity checksDefaultProfileInjector— replace unconditionalbuilder.build()with identity checksDefaultModelBuilder— pre-sizeArrayList, reuseHashMapacross phases, avoid redundantgetLifecycleBindingscallsDefaultPluginConfigurationExpander— add early-exit when no plugin executions existCommit 2: Additional hotspots (
b10d68d)DefaultModelValidator— hoist scope computation (stream pipeline + array allocation) before the dependency loopInternalSession.from()— addinstanceoffast-path before string concatenation in error messageDefaultInterpolator— shareHashSetfor cycle detection across all strings in a model (avoids ~550 allocations per module)DefaultInterpolator.unescape()— return input when no$$escapes found (avoids StringBuilder allocation)ReflectionValueExtractor— pre-compute method name inClassMapconstructor instead of per-lookupStringBufferDefaultProjectBuilder— use per-modulerootLocator.findRoot()(hoisting was incorrect due to per-moduleroot="true"support)Commit 3: MavenProject.hashCode() (
26b636f)Objects.hash(getGroupId(), getArtifactId(), getVersion())with inlined hash computation to avoid varargsObject[]allocation on every callCommit 4: Settings decryption fix (
094f0b5)indexOf('$')fast-path from sharedtransformer.vmtemplate intoDefaultModelInterpolatoras anonymous subclass overridetransformer.vmgenerates transformers for both interpolation AND settings decryption — the fast-path was skipping the decrypt function for encrypted passwords like{BteqUEnqHecHM7MZfnj9FwLcYbdInWxou1C929Txa0A=}since they don't contain$Commit 5: Root directory fix (
4a51d64)rootLocator.findRoot()hoisting — sub-modules can declareroot="true"in their POM, sofindRoot()must be called per-moduleJFR Allocation Progression
Test plan
mvn verifypasses locallyMavenITmng8379SettingsDecryptTestpasses (settings decryption not broken)MavenITmng7038RootdirTestpasses (per-module root directories work correctly)🤖 Generated with Claude Code