Flaky test-single-executable-application

Test

test-single-executable-application

Platform

macOS x64

Console output

00:23:42 not ok 2739 parallel/test-single-executable-application
00:23:42   ---
00:23:42   duration_ms: 120056.97200
00:23:42   severity: fail
00:23:42   exitcode: -15
00:23:42   stack: |-
00:23:42     timeout
00:23:42     Wrote single executable preparation blob to sea-prep.blob
00:23:42   ...

Build links

Additional information

I noticed it too. It became more prominent when I added another parallel e2e sea test in #47588. After I moved the test to sequential, the next fresh CI run didn't have any related failures.

It seems it gets stuck somewhere over here when run in parallel

node/src/node_sea.cc

Lines 160 to 163 in e8cfa95

    
           FPrintF(stderr, 
        
                   "Wrote single executable preparation blob to %s\n", 
        
                   config.output_path);

My hunch is that this has something to do with how the postject wasm blob uses memory - https://github.com/nodejs/postject/blob/3edd1dd1e0690167d7db0f502620e41888b2f82e/CMakeLists.txt#L23:

-sALLOW_MEMORY_GROWTH -sINITIAL_MEMORY=268435456 -sMAXIMUM_MEMORY=4294967296

From my testing, using LIEF purely through the c++ APIs is relatively faster than using it through wasm, which is what we're doing in postject.

Not sure at the moment how this could be improved though.

Also, cc @nodejs/single-executable

Can we just use a C++ addon instead of going throw WASM? I think postject locally is also surprisingly slow considering it's doing a pretty straight-forward thing...

@joyeecheung

@joyeecheung I had asked about whether implementing the injector in JS (i.e., without native deps) is necessary in @mhdawson's initial SEA PR and Jesse, one of the primary maintainers of pkg, said #42334 (comment):

Yes. Portability suffers and complexity greatly increases if we have to depend on some native utilities. Also, it becomes harder to promote the feature if user/developer found it hard to use.

So IIUC, if we do this, we would need to maintain prebuilt LIEF binaries for all supported platforms and architectures, even though LIEF's code doesn't really do platform-specific stuff. But since postject uses wasm, we only need to maintain a single blob and that works on all platforms.

@RaisinTen

I think the extra overhead/complications of having to maintain prebuilt LIEF binaries, or building them as part of postject would add significant work/complications.

It might be better to leave as is since @RaisinTen mentioned that the test issue can be addressed by a move to sequential and look to see if we can understand why it's slower than the native implementation. Maybe there is something in our uvwasi implementation that needs to be optimized or maybe LIEF is doing reads/writes in a way that magnifies overhead in WASM (as the only thing I can think of being slower in WASM is the read/write of the binary)

sea: add option to disable the experimental SEA warning

Btw, I have found out the bottleneck that was causing Postject to be so slow in nodejs/postject#85 and I have created a PR to fix it - nodejs/postject#86.

Navigation Menu

Repository navigation

Flaky test-single-executable-application #47741

Test

Platform

Console output

Build links

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development


	FPrintF(stderr,
	"Wrote single executable preparation blob to %s\n",
	config.output_path);

Flaky test-single-executable-application #47741

Description

Test

Platform

Console output

Build links

Additional information

Activity

RaisinTen commented on Apr 27, 2023

debadree25 commented on Apr 27, 2023

RaisinTen commented on Apr 27, 2023

joyeecheung commented on Apr 27, 2023

RaisinTen commented on Apr 28, 2023

mhdawson commented on May 2, 2023

RaisinTen commented on May 16, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions