Sitelet https://github.com/nodejs/node/issues/46140
Skip to content

Memory usage spikes with source maps (--enable-source-maps) from v18.10.0 onwards #46140

Closed
#46225
Closed
Memory usage spikes with source maps (--enable-source-maps) from v18.10.0 onwards#46140
#46225
Labels
memoryIssues and PRs related to Node.js memory management or memory footprint.source mapsIssues and PRs related to source map support.

Description

@joelmukuthu
joelmukuthu
opened on Jan 9, 2023

Version

18.10.0

Platform

5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

UPDATE: The steps below attempt to create a minimal reproduction of the issue, rather unsuccessfully, since the issue seems to be caused by large source maps and not merely throwing/handing an error. See #46140 (comment) for more info.

Obsolete minimal reproduction

Copy the following code blocks and save them as main.js and main.js.map and then run node --enable-source-maps main.js:

main.js:

// main.ts
var run = (withError) => {
  console.log(
    "Memory usage before (MB):",
    process.memoryUsage().heapUsed / 1024 / 1024
  );
  if (withError) {
    try {
      throw new Error("foo");
    } catch (e) {
      e.stack;
    }
  }
  console.log(
    "Memory usage after (MB):",
    process.memoryUsage().heapUsed / 1024 / 1024
  );
};
console.log("\nWithout error:");
run(false);
console.log("\nWith error:");
run(true);
console.log("\nWithout error:");
run(false);
console.log("\nWith error:");
run(true);
//# sourceMappingURL=main.js.map

main.js.map:

{
  "version": 3,
  "sources": ["main.ts"],
  "sourcesContent": ["const run = (withError) => {\n  console.log(\n    \"Memory usage before (MB):\",\n    process.memoryUsage().heapUsed / 1024 / 1024\n  );\n\n  if (withError) {\n    try {\n      throw new Error(\"foo\");\n    } catch (e) {\n      e.stack;\n    }\n  }\n\n  console.log(\n    \"Memory usage after (MB):\",\n    process.memoryUsage().heapUsed / 1024 / 1024\n  );\n};\n\nconsole.log(\"\\nWithout error:\");\nrun(false);\n\nconsole.log(\"\\nWith error:\");\nrun(true);\n\nconsole.log(\"\\nWithout error:\");\nrun(false);\n\nconsole.log(\"\\nWith error:\");\nrun(true);\n"],
  "mappings": ";AAAA,IAAM,MAAM,CAAC,cAAc;AACzB,UAAQ;AAAA,IACN;AAAA,IACA,QAAQ,YAAY,EAAE,WAAW,OAAO;AAAA,EAC1C;AAEA,MAAI,WAAW;AACb,QAAI;AACF,YAAM,IAAI,MAAM,KAAK;AAAA,IACvB,SAAS,GAAP;AACA,QAAE;AAAA,IACJ;AAAA,EACF;AAEA,UAAQ;AAAA,IACN;AAAA,IACA,QAAQ,YAAY,EAAE,WAAW,OAAO;AAAA,EAC1C;AACF;AAEA,QAAQ,IAAI,kBAAkB;AAC9B,IAAI,KAAK;AAET,QAAQ,IAAI,eAAe;AAC3B,IAAI,IAAI;AAER,QAAQ,IAAI,kBAAkB;AAC9B,IAAI,KAAK;AAET,QAAQ,IAAI,eAAe;AAC3B,IAAI,IAAI;",
  "names": []
}

On my machine, NodeJS v18.11.0 produces:

Without error:
Memory usage before (MB): 5.0794219970703125
Memory usage after (MB): 5.0850372314453125

With error:
Memory usage before (MB): 5.0877227783203125
Memory usage after (MB): 5.37274169921875

Without error:
Memory usage before (MB): 5.37542724609375
Memory usage after (MB): 5.377166748046875

With error:
Memory usage before (MB): 5.379974365234375
Memory usage after (MB): 5.411712646484375

While NodeJS v18.10.0 produces:

Without error:
Memory usage before (MB): 5.038932800292969
Memory usage after (MB): 5.044548034667969

With error:
Memory usage before (MB): 5.047233581542969
Memory usage after (MB): 5.0955810546875

Without error:
Memory usage before (MB): 5.0982666015625
Memory usage after (MB): 5.100006103515625

With error:
Memory usage before (MB): 5.102813720703125
Memory usage after (MB): 5.1345062255859375

When an error is thrown, the increase in memory usage on v18.11.0 spikes a bit higher than on v18.10.0. The same behaviour can be observed in later versions after v18.11.0.

For reference, without --enable-source-maps, this is the output on v18.11.0:

Without error:
Memory usage before (MB): 5.0529632568359375
Memory usage after (MB): 5.0585479736328125

With error:
Memory usage before (MB): 5.0612640380859375
Memory usage after (MB): 5.067131042480469

Without error:
Memory usage before (MB): 5.069816589355469
Memory usage after (MB): 5.071556091308594

With error:
Memory usage before (MB): 5.074363708496094
Memory usage after (MB): 5.080055236816406

And on v18.10.0:

Without error:
Memory usage before (MB): 5.012535095214844
Memory usage after (MB): 5.018119812011719

With error:
Memory usage before (MB): 5.020835876464844
Memory usage after (MB): 5.026702880859375

Without error:
Memory usage before (MB): 5.029388427734375
Memory usage after (MB): 5.0311279296875

With error:
Memory usage before (MB): 5.033935546875
Memory usage after (MB): 5.0396270751953125

How often does it reproduce? Is there a required condition?

It's consistent.

What is the expected behavior?

That memory usage doesn't spike when an error is handled by the application.

What do you see instead?

Memory usage spikes (ref: #46140 (comment)).

Additional information

No response

Activity

  1. joelmukuthu
    changed the title [-]Memory leak with source maps (`--enable-source-maps`) from v18.11.0[/-] [+]Memory leak with source maps (`--enable-source-maps`) from v18.11.0 onwards[/+] on Jan 9, 2023
  2. VoltrexKeyva
    added
    source mapsIssues and PRs related to source map support.
    memoryIssues and PRs related to Node.js memory management or memory footprint.
    on Jan 10, 2023

  3. bnoordhuis commented on Jan 10, 2023

    @bnoordhuis
    bnoordhuis
    on Jan 10, 2023
    Member

    The difference looks rather insignificant. I don't think you can say marginally higher memory usage indicates a memory leak. It might, but it probably isn't.

    Is this a problem for you somehow or just something you noticed?

  4. joelmukuthu commented on Jan 10, 2023

    @joelmukuthu
    joelmukuthu
    on Jan 10, 2023
    Author

    Yes, memory leak might be a bit harsh but on a project I work on, memory usage spikes by about 150MB whenever an error is handled by the application. When that application is run in a container on Kubernetes, whenever an error is handled, the container instantly hits its memory limit and is restarted by K8s. So it's rather serious and blocking upgrading to newer versions.

    The application's code is bundled into a single JS file, with only a few external dependencies, so stack traces aren't crazy long. It appears there might be an underlying issue in v18.11.0, it was just hard to prove with a minimal reproduction.

  5. bnoordhuis commented on Jan 10, 2023

    @bnoordhuis
    bnoordhuis
    on Jan 10, 2023
    Member

    memory usage spikes by about 150MB whenever an error is handled by the application

    That's pretty severe. How does pre-v18.11.0 compare? No spike, or spiky but not as extreme? How big are the source maps?

  6. joelmukuthu commented on Jan 11, 2023

    @joelmukuthu
    joelmukuthu
    on Jan 11, 2023
    Author

    How does pre-v18.11.0 compare?

    The app is currently running on v18.9.1 and there's no spike there -- memory consumption when an error is handled is just slightly larger than for the no-error scenario.

    How big are the source maps?

    App script: main.js -> 9.7MB
    Source maps: main.js.map -> 12MB

  7. bnoordhuis commented on Jan 11, 2023

    @bnoordhuis
    bnoordhuis
    on Jan 11, 2023
    Member

    Can you test with v18.10.0? I don't think there are any changes between v18.10.0 and v18.11.0 that could explain such a jump but between v18.9.1 and v18.10.0 there's at least a2eb55a.

  8. joelmukuthu commented on Jan 12, 2023

    @joelmukuthu
    joelmukuthu
    on Jan 12, 2023
    Author

    I think you might be on to something:

    Version Initial (MB) 1st error (MB) 2nd error (MB) 3rd error (MB) 4th error (MB) 5th error (MB)
    18.9.1 631 633.8 634 (then 633) 633.4 633.5 633.7
    18.10.0 675.2 857.9 999.1 1035 (then 752) 786 906
    18.11.0 708.2 728 785 737 865 1012

    I ran rudimentary tests and got these numbers from docker stats so they weren't exactly controlled tests. As a control, I also tried a few requests where I didn't generate errors for each of the versions and memory usage stayed rather consistent, no spikes there.

    It's interesting that my minimal reproduction (original post) didn't show spikes on 18.10.0!

  9. bnoordhuis commented on Jan 12, 2023

    @bnoordhuis
    bnoordhuis
    on Jan 12, 2023
    Member

    @alan-agius4 @aduh95 @legendecas a2eb55a seems to cause excessive memory use. I can't tell if the commit introduced it, or if the issue was already present but latent. You should chime in. :-)

  10. joelmukuthu
    changed the title [-]Memory leak with source maps (`--enable-source-maps`) from v18.11.0 onwards[/-] [+]Memory leak with source maps (`--enable-source-maps`) from v18.10.0 onwards[/+] on Jan 12, 2023
  11. joelmukuthu
    changed the title [-]Memory leak with source maps (`--enable-source-maps`) from v18.10.0 onwards[/-] [+]Memory usage spikes with source maps (`--enable-source-maps`) from v18.10.0 onwards[/+] on Jan 14, 2023

  12. legendecas commented on Jan 15, 2023

    @legendecas
    legendecas
    on Jan 15, 2023
    Member

    Thanks for reporting! I'll create a large JS file with about source map of 10M+ size to verify this issue.

  13. nodejs-github-bot
    closed this as completedin #46225on Jan 22, 2023

  14. bolt-juri-gavshin commented on May 22, 2023

    @bolt-juri-gavshin
    bolt-juri-gavshin
    on May 22, 2023

    I think this issue was marked as completed by mistake.
    If I understand everything correctly, current LTS (v18.16.0) still has the issue...

  15. bnoordhuis commented on May 22, 2023

    @bnoordhuis
    bnoordhuis
    on May 22, 2023
    Member

    The backport of the fix was blocked on other changes not backporting cleanly but that's been resolved now. I've pinged @juanarbol about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    memoryIssues and PRs related to Node.js memory management or memory footprint.source mapsIssues and PRs related to source map support.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions